Shared, Isolated Dev Environments with LXC

Our team has been using Gitpod for development, but with the recent deprecation of Gitpod (Classic), we migrated to GitHub Codespaces. Other than Gitpod is cheaper (at that time) with flat pricing instead of pay per use, the reason we don't use Codespaces because it didn't support docker-in-docker which is crucial to our dev setup.

But now Codespaces is based dev container and dind support can be added via dev container features settings. While codespaces works fine, I still prefer to work on terminal and we have a dev server (let's call it kdev) with 64GB RAM lying around, so I looks into using LXC as my dev environment.

Docker Compose on a Shared Host

Our project's local development environment is standardized around Docker Compose to bring up all necessary dependencies:

• Services: PostgreSQL (port 5432), Redis (port 6379), and several other back-end services, all using their standard ports.
• Workflow: A simple, repeatable process using make commands:
  • make up: Runs docker compose up to start the service stack.
  • make dev: Populates the development database with initial data.
  • make run: Starts the main Django development server on port 8000.

The inherent problem with this setup on a shared server is the use of those standard ports. If one developer claims a port, everyone else is blocked. This is what made cloud environments (where each dev gets an isolated machine/container) so necessary. LXC provides that same level of isolation right on kdev.

The LXC Solution: Isolated Networks

LXC (LinuX Containers) provides OS-level virtualization. Unlike full VMs, containers are lightning-fast and resource-efficient, but unlike Docker application containers, they act like full Linux machines with their own isolated network stack.

By creating an LXC container for each project, the ports inside my container (5432, 6379, 8000) are completely independent of the ports in another developer’s container. Problem solved!

Setting up the LXC Development Flow

1. Container Access: I log in to my isolated environment using SSH, leveraging kdev as a jump host:

   bash ssh container-ip -J kdev -l ubuntu

   This gives me a clean, dedicated Ubuntu environment every time.

2. Persistent Terminal Sessions with GNU Screen: Since I'm working remotely, I use GNU Screen to maintain persistence. I run my long-running services in separate screen windows:

   • make up in one window.
   • make run in a second window. I can detach the session (Ctrl+A then d) and disconnect my SSH, knowing my development server is still running.

3. Terminal-Focused Tooling: My environment is lean and mean. I use Vim for all my coding, augmented by AI assistance through Claude Code or GitHub Copilot CLI right in the terminal, keeping my hands on the keyboard and my focus on the code.

Sharing and Demoing with Cloudflare Zero Trust Tunnel

A major requirement is sharing the running development site for team testing or quick demos. Exposing a random port on kdev is cumbersome and insecure.

We use Cloudflare Zero Trust Tunnels to create a secure, public HTTPS endpoint directly to the internal port 8000 of my isolated LXC container.

To automate this setup, I wrote a simple Python script (cftunnel.py):

python3 cftunnel.py --url http://localhost:8000 \
    --domain kdev.io \
    --subdomain some-weird-name \
    --access-email='*@kafkai.com'

This command instantly provides a secure URL like https://some-weird-name.kdev.io/. Crucially, access is protected with a secure OTP login, restricted only to emails matching the @kafkai.com domain.

The Power of LXC Copy: Instant Environment Snapshots

For long-lived demo or testing sites, the cloning capability of LXC is a huge win. If I need to show a feature that should remain static for a week, but I need to continue development, I don't have to worry about polluting the demo environment.

I simply use the lxc copy command to duplicate the current container. This creates an exact clone, including the running state, database, and all services, in seconds.

• Original Container: Stays frozen as the pristine demo site.
• New Container: I immediately jump back into the new copy to continue my work from the exact same point.

There's a variation to this setup where we use code-server to provide in-browser VSCode experience for those not so keen on using Vim and terminal. This involved some nginx setup as well to forward the port for code-server itself. Maybe a topic for next blog post?

We’ve been running isolated LXC containers on our spare server (kdev) with 64GB of RAM. That means no more port conflicts, full control over our dev environment, and zero extra cost compared to cloud alternatives. Docker Compose, LXC, and Cloudflare tunnels give us everything we need to develop, test, and share projects securely, all from the terminal.

Stay tuned for the next post, where I’ll share either the step-by-step LXC setup or the Python script that automates Cloudflare tunnels!

Keep an eye on our blog, and don’t forget to share it with others who might find it useful.