JustToThePoint English Website Version
JustToThePoint en español

Automating a Python Development Environment in Proxmox Arch Linux Containers

Irony is wasted on the stupid, Oscar Wilde

Complex Analysis

Setting up a robust Python development environment in a homelab can be greatly simplified using Proxmox VE and Linux Containers (LXC). Proxmox VE is a popular open-source hypervisor for homelabs, offering an easy way to manage both virtual machines and lightweight containers on a single server.

In this article, we will automate the entire process of setting up a Python development environment within an Arch Linux container on Proxmox. Instead of polluting your Proxmox host or main server with development libraries, you can spin up an isolated Arch Linux container that contains all the tools and dependencies you need. This isolation means you avoid the “it works on my machine” syndrome, meaning that your Python applications behave consistently across different systems.

Arch Linux is known for being lightweight and up-to-date. Our goal is to simplify the transfer from local machines, enhance management, and streamline the deployment of Python applications.

This is the third article in our series. If you haven’t already, consider reading the first two articles:

Creating and configuring a Python deployment container in Proxmox

Step 1. Create and configure the container, install Python, and create the environment.

sh /home/nmaximo7/homelab/mydockers/custom-arch-python.sh

#!/bin/bash

# Variables
CTID=308 # Container ID (must be unique)
OSTEMPLATE="archlinux-base_20240911-1_amd64.tar.zst" # Arch linux template file
TEMPLATE_STORAGE="local" # Storage location for the template
CONTAINER_STORAGE="mypool" # Storage location for the container's disk
DISK_SIZE="80" # Disk size in GB for container rootfs
PASSWORD="YOUR-PASSWORD" # Root password
MEMORY=4096 # Allocate 4096 MB (4 GB) RAM to the container
CORES=2 # Allocate 2 CPU cores to container
BRIDGE="vmbr0"  # Bridge name for networking
IPADDRESS="192.168.1.38" # Static IP address for the container
GATEWAY="192.168.1.1" # LAN Gateway for network configuration
CIDR="/24" # CIDR notation for the subnet
# Adjust if not 255.255.255.0
HOSTNAME="ArchPython" # Hostname for the container

# 1. Check if the container already exists; if it does, stop and remove it
if pct status $CTID &>/dev/null; then
	echo "Container $CTID exists. Stopping the container."
	pct stop $CTID
	sleep 2 # Wait for 2 seconds to allow for a graceful stop
	echo "Proceeding with deletion of container $CTID."
	pct destroy $CTID
else echo "Container $CTID does not exist."
fi

# 2. Before creating the container, you need the Arch Linux LXC template on your Proxmox server.
# Download the Arch Linux template if it's not already present
if ! pveam list $TEMPLATE_STORAGE | grep -q $OSTEMPLATE; then
  echo "Downloading Arch template..."
  pveam download $TEMPLATE_STORAGE $OSTEMPLATE
else
  echo "Arch template already exists."
fi

# 3. Create a privileged container using the specified Arch template and settings.
# pct create $CTID $TEMPLATE is the core command to create a new container...
# where $CTID is the numeric ID we chose and $TEMPLATE is the storage and filename of the Arch Linux template
# Example: local:vztmpl/archlinux-base_20240911-1_amd64.tar.zst which points to the template we [previously] downloaded on the local storage.
# -hostname $HOSTNAME: Sets the container’s hostname
# -cores $CORES: Allocates CPU cores to the container
# -memory $MEMORY: Allocates memory (RAM) to the container in MB.
# -rootfs mypool:80: Specifies storage for the container’s root filesystem and its size.
# -net0 name=eth0,bridge=$BRIDGE,ip=$IP: Configures the network interface for the container.
# Using a bridge to vmbr0 ensures the container is directly on your LAN (which is convenient for SSH and development).
# -unprivileged 1: Unprivileged container flag. We set this to 1 (true) to create an unprivileged container, which is the default and recommended for security.
# (Optional) -features nesting=1: Enables the nesting feature, which allows certain capabilities inside the container, such as the ability to run Docker or nested containers if needed.
# -password $ROOTPW: Sets the root password for the container.
pct create $CTID $TEMPLATE_STORAGE:vztmpl/$OSTEMPLATE \
  --storage $CONTAINER_STORAGE \
  --rootfs $CONTAINER_STORAGE:$DISK_SIZE \
  --password $PASSWORD \
  --hostname $HOSTNAME \
  --memory $MEMORY \
  --cores $CORES \
  --net0 name=eth0,bridge=$BRIDGE,ip=$IPADDRESS$CIDR,gw=$GATEWAY \
  --unprivileged 0 # Set to 1 for unprivileged containers

# Once done, the container will be registered on the Proxmox host.
# Verify its creation by running pct list | grep $CTID
# root@myserver:~# pct list | grep 308
# 308        running                 ArchPython

# 4. Starting and Accessing the Container
# Set the container not to start automatically on boot
pct set $CTID --onboot 0

# (Optional) Unconfine AppArmor for additional flexibility
echo "lxc.apparmor.profile: unconfined" >> /etc/pve/lxc/$CTID.conf

# After creation, the container exists but is not yet running. We need to start it to proceed with configuration inside it.
pct start $CTID

# (Optional) Wait a few seconds for the container to boot up.
# If you configured networking with DHCP, the container will request an IP at startup, check the IP by running:
# root@myserver:~# pct exec 308 -- ip -4 addr show eth0
# 2: eth0@if19:  mtu 1500 qdisc noqueue state UP group default qlen 1000 link-netnsid 0
#    inet 192.168.1.38/24 brd 192.168.1.255 scope global eth0
#       valid_lft forever preferred_lft forever
sleep 5

# 5. Initial Container Configuration and System Updates
# Now that the container is running, we need to configure the Arch Linux system inside it.
# Initialize and populate pacman keyring (required for package verification)
pct exec $CTID -- pacman-key --init
pct exec $CTID -- pacman-key --populate archlinux

Arch Linux uses package signing, and in a fresh environment you need to initialize the pacman keyring. pacman-key ‐‐init creates the GPG keyring for package verification, and pacman-key ‐populate archlinux populates it with the official Arch signing keys. Otherwise, skipping this critical step can lead to error when pacman tries to verify package signatures.

# Update package database and refresh Arch keyring package to get latest keys
pct exec $CTID -- pacman -Sy archlinux-keyring --noconfirm

# (Optional) Refresh the pacman keys from keyservers (adds extra assurance)
pct exec $CTID -- pacman-key --refresh-keys

# Full system upgrade (update all packages to latest versions)
pct exec $CTID -- pacman -Syu --noconfirm

# Install reflector and update mirror list for faster downloads
pct exec $CTID -- pacman -S --noconfirm reflector
pct exec $CTID -- reflector --country Spain,France,Germany \
    --latest 20 \
    --protocol http,https \
    --save "/etc/pacman.d/mirrorlist" \
    --sort rate

# Install base development packages and essential tools
pct exec $CTID -- pacman -Syu --noconfirm base-devel git

# Install additional tools for productivity
pct exec $CTID -- pacman -S --noconfirm eza zoxide fd bat ripgrep bc

# Install system utilities, help (tldr), and fuzzy finder (fzf)
pct exec $CTID -- pacman -S --noconfirm fzf wl-clipboard lazygit jq rsync man-db tldr unzip plocate glibc sudo wget curl

# Install various system utilities for enhanced functionality
pct exec $CTID -- pacman -S --noconfirm fastfetch vim bpytop cmatrix starship figlet fortune-mod

# Install glibc and glibc-locales for locale generation support
pct exec $CTID -- pacman -S --noconfirm glibc glibc-locales

# By default, Arch Linux containers may have no locale generated, which can lead to warnings or unicode issues.
# We set the system locale to US English UTF-8.
# The first command uses sed to uncomment the en_US.UTF-8 UTF-8 line in /etc/locale.gen
pct exec $CTID -- sed -i 's/^#\(en_US.UTF-8 UTF-8\)/\1/' /etc/locale.gen
# Next, we echo LANG=en_US.UTF-8 into /etc/locale.conf, which will make this the default locale for the system.
pct exec $CTID -- bash -c "echo 'LANG=en_US.UTF-8' > /etc/locale.conf"
# Finally, locale-gen compiles the locales, generating the en_US.UTF-8 locale data.
# After this, the system’s locale is configured to UTF-8 English.
pct exec $CTID -- locale-gen

# Set the timezone to Europe/Madrid by default
pct exec $CTID -- ln -sf /usr/share/zoneinfo/Europe/Madrid /etc/localtime

# Configure the keyboard layout
pct exec $CTID -- echo "KEYMAP=us" >> /etc/vconsole.conf

# 6. Installing Essential Packages
# The Arch template is minimal, so you’ll likely want to install some additional packages to make the environment user-friendly and ready for development.
# Install additional libraries for GUI applications
# Arch’s base-devel group includes GCC, make, and other build essentials.
pct exec $CTID -- pacman -S --noconfirm alsa-lib gtk3 libcups libxss libxtst nss ttf-liberation xdg-utils geany

# Install extra tools for various tasks
pct exec $CTID -- pacman -S --noconfirm cowsay ripgrep bat eza htop feh gedit curl
pct exec $CTID -- pacman -S --noconfirm ranger ddgr surfraw sudo

# Install Python and essential libraries are installed,
# along with virtualenv for creating isolated Python environments
pct exec $CTID -- pacman -S --noconfirm python python-pip python-requests python-beautifulsoup4 python-pytz
pct exec $CTID -- pacman -S --noconfirm python-virtualenv

# 7. User Creation and Sudo Setup
# Create the user 'devuser', e.g., nmaximo7.
# The -m flag creates a home directory for the user (e.g., /home/devuser).
# The user is added to the wheel group (-G wheel), which on Arch is typically configured for admin privileges.
pct exec $CTID -- useradd -m -G wheel nmaximo7
# We use a chpasswd command via bash -c to avoid interactive input.
# This echoes "username:password" into chpasswd, which sets the password for the new user.
pct exec $CTID -- echo "nmaximo7:Anawim" | chpasswd
# We append a NOPASSWD rule for nmaximo7 which grant sudo access without a password prompt.
pct exec $CTID -- echo "nmaximo7 ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

# Set up environment variables for language and locale settings
# Set the keyboard layout to US.
pct exec $CTID -- echo "KEYMAP=us" >> /etc/vconsole.conf
# Set the default language to English (United States) for the user nmaximo7.
pct exec $CTID -- echo "export LANG=en_US.UTF-8" >> /home/nmaximo7/.bashrc
# Set User Language Preference
pct exec $CTID -- echo "export LANGUAGE=en_US:en" >> /home/nmaximo7/.bashrc

# Create a backup directory for storing Python files (if it doesn't already exist)
pct exec $CTID -- sh -c "[ ! -d "/home/nmaximo7/backup" ] && mkdir -p /home/nmaximo7/backup"

# Set up a Python development environment for the user nmaximo7 within the specified container.
# Create a directory for Python projects
pct exec $CTID -- sh -c "cd /home/nmaximo7 && mkdir myNewPython"
# Initializes a virtual environment for Python development.
# This command navigates to the myNewPython directory, initializes a Python virtual environment named .venv, activates it, and upgrades pip to the latest version.
pct exec $CTID -- sh -c "cd /home/nmaximo7/myNewPython && python -m venv .venv && source .venv/bin/activate && pip install --upgrade pip"
# Install Required Packages
# This command activates the virtual environment again and installs two Python packages:
# python-dotenv (for managing environment variables) and requests (for making HTTP requests).
pct exec $CTID -- sh -c "cd /home/nmaximo7/myNewPython && source .venv/bin/activate && pip install python-dotenv && pip install requests"
# It changes the ownership of the myNewPython directory and its contents to the user nmaximo7, ensuring that the user has the necessary permissions to access and modify the files.
pct exec $CTID -- sh -c "chown -R nmaximo7:nmaximo7 /home/nmaximo7/myNewPython"

# Install SSH and rsync for remote access and file transfer capabilities from Proxmox node to the container
pct exec $CTID -- pacman -S --noconfirm openssh rsync
pct exec $CTID -- systemctl enable sshd
pct exec $CTID -- systemctl start sshd

# Enable root login (if needed) in /etc/ssh/sshd_config
pct exec $CTID -- sed -i 's/#\?PermitRootLogin.*/PermitRootLogin yes/' /etc/ssh/sshd_config
# Modify the SSH configuration on our Proxmox container to enable password authentication
pct exec $CTID -- sed -i 's/#\?PasswordAuthentication.*/PasswordAuthentication yes/' /etc/ssh/sshd_config
# After running these commands, don't forget to restart the SSH service inside your container to apply the changes.
pct exec $CTID -- systemctl restart sshd

# Display welcome message and system information
pct exec $CTID -- echo "Welcome master"
pct exec $CTID -- fastfetch
pct exec $CTID -- echo "=============DISK USAGE================"
pct exec 308 -- df -h | grep -E '^Filesystem|^mypool|^/'
pct exec $CTID -- echo "=============MEMORY USAGE=============="
pct exec $CTID -- free -h
pct exec $CTID -- echo "=========TOP 5 CPU PROCESSES==========="
pct exec $CTID -- ps -eo pid,comm,%mem,%cpu --sort=-%cpu | head -n 6
pct exec $CTID -- sh -c "fortune | cowsay"

# Fetch weather information
pct exec $CTID -- curl -s 'wttr.in/Malaga?format=3'

# Display current date and time
pct exec $CTID -- date '+%d/%m/%Y %H:%M:%S'

# Display calendar
pct exec $CTID -- cal
echo "Client (NixOS Desktop Computer)"
echo "scp -r /home/nmaximo7/dotfiles/proxmox/myNewPython root@192.168.1.38:/root/terraform/"
echo "Proxmox. Post Installation"
echo "sh /home/nmaximo7/homelab/mydockers/post-custom-arch-python.sh "

Next, Set up password-less SSH access to the Proxmox server

Transferring Python Project from Local Machine to Proxmox VM

When your Proxmox (e.g., 192.168.1.33) is reachable directly (that is, you can SSH into 192.168.1.33 from your NixOS Desktop computer), copy the necessary Python files (Python code files .py, .env (environment variables), requirements.txt, assets dictory, and any other necessary files from your destop/laptop to Proxmox:

# First, make sure the target directory (e.g., /root/terraform/) exists on the remote server:
# scp -r /path/on/NixOS/myNewPython/ root@192.168.1.33:/path/on/Proxmox/backup/
ssh root@192.168.1.33 "mkdir -p /root/terraform/"

# Use the Legacy SCP Protocol (-O flag)
scp -r -O /home/nmaximo7/dotfiles/proxmox/myNewPython root@192.168.1.33:/root/terraform/

The scp (secure copy) command is a great way to transfer files between your NixOS Desktop and your Proxmox server. -r stands for recursive, which is used to copy directories and their contents. Make sure you’ve already set up password-less SSH access, as this will streamline the process.

Transferring Python Project from ProxMox to your VM

Copy from ProxMox the necessary files to your VM: scp -r /root/terraform/myNewPython/ root@192.168.1.38:/home/nmaximo7/backup/

Post-installation configuration

Post-installation configuration in the VM (192.168.1.38): sh /home/nmaximo7/homelab/mydockers/post-custom-arch-python.sh

This script automates the post-installation configuration for a Python project running within a Proxmox container. It handles the setup of SSH keys for secure access, transfers the project files, and prepares the Python environment for development.

#!/bin/bash
# nvim post-custom-arch-python.sh

CTID="308" # Container ID for reference
VM_IP="192.168.1.38" # IP address of the Container
SOURCE_PATH="/root/terraform/myNewPython/" # Local path of the source directory
DEST_PATH="/home/nmaximo7/backup/" # Destination path on the container

# 0. Start the container only if it not already running
if [ "$(pct status $CTID | awk '{print $2}')" != "running" ]; then
  pct start $CTID
fi
# 1.  Generate an SSH key if it does not already exist
echo "Generate an SSH key if it does not already exist"
if [ ! -f ~/.ssh/id_rsa ]; then
    ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
     # Create a new RSA key without a passphrase
fi

# 2. Copy the public key to the VM (you'll be asked for the VM's root password once).
echo "Copy the public key to the VM"
ssh-copy-id "root@${VM_IP}"
# Install the public key on the Container for passwordless SSH login

# 3. Transfer the directory to the VM.
echo "Transfer the directory to the VM"
scp -r "${SOURCE_PATH}" "root@${VM_IP}:${DEST_PATH}"
# Securely copy the entire (-r) source directory ("/root/terraform/myNewPython/") from the local system (ProxMox) to the container's destination path ("/home/nmaximo7/backup/")

# After the file transfer, the script prints a message confirming that the files were copied successfully from Proxmox to the destination on the container.
echo "Done. Files from ${SOURCE_PATH} copied to ${DEST_PATH} on VM ${VM_IP}."

# 4. Copy the necessary files (Python source files *.py, environment file .env, requirements file (requirements.txt), and assets directory) from the VM's backup directory to the specified directory in the container.
# The shell invoked by pct exec does not expand the wildcard (*.py, e.g., if the default shell is not bash) unsless we force the use of bash -c
pct exec 308 -- bash -c 'cp /home/nmaximo7/backup/myNewPython/*.py /home/nmaximo7/myNewPython/'
pct exec $CTID -- cp /home/nmaximo7/backup/myNewPython/.env /home/nmaximo7/myNewPython
pct exec $CTID -- cp /home/nmaximo7/backup/myNewPython/requirements.txt /home/nmaximo7/myNewPython
pct exec $CTID -- cp -r /home/nmaximo7/backup/myNewPython/assets /home/nmaximo7/myNewPython

# 5. Activate the virtual environment and install required Python packages listed in requirements.txt
# The command ensures that the latest version of pip is installed before installing the project dependencies.
pct exec $CTID -- sh -c "cd /home/nmaximo7/myNewPython && source .venv/bin/activate && pip install --upgrade pip && pip install -r /home/nmaximo7/myNewPython/requirements.txt"

# 6. Enhance security
pct exec $CTID -- sed -i 's/^#PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config
pct exec $CTID -- sed -i 's/^#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config
pct exec $CTID -- systemctl restart sshd

Activate our Python environment and run our code.

# Enter our container
pct enter 308
# Navigate to the project directory
cd /home/nmaximo7/myNewPython/
# Activate the virtual environment
source .venv/bin/activate
# Run the Python script
python greeting.py
Bitcoin donation

JustToThePoint Copyright © 2011 - 2025 Anawim. ALL RIGHTS RESERVED. Bilingual e-books, articles, and videos to help your child and your entire family succeed, develop a healthy lifestyle, and have a lot of fun. Social Issues, Join us.

This website uses cookies to improve your navigation experience.
By continuing, you are consenting to our use of cookies, in accordance with our Cookies Policy and Website Terms and Conditions of use.