How to Set Up Automated Testing for AI-Built Sites on Linux

Why Testing AI-Generated Code Matters More Than You Think

When you're working with an AI pair programmer like Claude Code to build your site, the code ships fast. That's the whole point. But speed without confidence is how you end up debugging in production at 2 a.m.

AI-generated code is often clean and well-structured, but it's not infallible. The AI might misunderstand a requirement, miss an edge case, or write logic that works in isolation but breaks when integrated with your database or third-party APIs. Without automated tests, you won't know until a user finds the bug.

The good news: setting up automated testing on a Linux hosting environment is straightforward, and it catches problems before they reach production. This guide walks you through the essentials.

Start with Unit Tests — Test Individual Functions

Unit tests are the foundation. They verify that a single function or method does what you expect, in isolation.

For Python apps: Use pytest. It's simple, readable, and integrates well with CI pipelines.

# example_test.py
import pytest
from app import calculate_discount

def test_calculate_discount_standard():
    assert calculate_discount(100, 0.1) == 90

def test_calculate_discount_zero_price():
    assert calculate_discount(0, 0.1) == 0

def test_calculate_discount_invalid_rate():
    with pytest.raises(ValueError):
        calculate_discount(100, 1.5)  # discount > 100%

Run tests locally: pytest. That's it.

For Node.js apps: Jest is the standard. It's zero-config and comes with built-in assertion matchers.

// math.test.js
const { add, multiply } = require('./math');

test('adds two numbers', () => {
  expect(add(2, 3)).toBe(5);
});

test('multiplies two numbers', () => {
  expect(multiply(4, 5)).toBe(20);
});

Run tests: npm test.

Pro tip: Ask Claude Code to write tests alongside the feature. It often generates solid test cases and catches its own assumptions. If it doesn't, you've found your first integration point to verify.

Add Integration Tests — Verify Components Talk to Each Other

Unit tests are great, but they don't catch integration bugs. Integration tests verify that your API endpoints work, your database queries return the right data, and your frontend can actually call your backend.

For a Python Flask or FastAPI app:

# test_api.py
import pytest
from app import create_app, db
from models import User

@pytest.fixture
def client():
    app = create_app()
    app.config['TESTING'] = True
    app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///:memory:'
    
    with app.app_context():
        db.create_all()
        yield app.test_client()
        db.session.remove()
        db.drop_all()

def test_create_user(client):
    response = client.post('/api/users', json={
        'name': 'Alice',
        'email': 'alice@example.com'
    })
    assert response.status_code == 201
    assert response.json['email'] == 'alice@example.com'

def test_get_user(client):
    # Create a user first
    client.post('/api/users', json={'name': 'Bob', 'email': 'bob@example.com'})
    # Now fetch it
    response = client.get('/api/users/1')
    assert response.status_code == 200
    assert response.json['name'] == 'Bob'

For a Node.js Express app:

// test/api.test.js
const request = require('supertest');
const app = require('../app');
const db = require('../db');

beforeAll(async () => {
  await db.connect(':memory:');
  await db.migrate();
});

afterEach(async () => {
  await db.clear();
});

describe('POST /api/users', () => {
  it('creates a new user', async () => {
    const res = await request(app)
      .post('/api/users')
      .send({ name: 'Charlie', email: 'charlie@example.com' });
    
    expect(res.status).toBe(201);
    expect(res.body.email).toBe('charlie@example.com');
  });
});

Integration tests use a real (or in-memory) database and make actual HTTP calls. They're slower than unit tests, but they catch the bugs that matter.

Set Up a CI Pipeline to Run Tests Automatically

Running tests manually is fine during development, but you want them to run automatically every time you push code. That's where CI (continuous integration) comes in.

GitHub Actions is free and built into GitHub. Here's a minimal example:

# .github/workflows/test.yml
name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    
    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: testpass
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
      
      - name: Run tests
        run: pytest
        env:
          DATABASE_URL: postgresql://postgres:testpass@localhost:5432/testdb

Every push to your repository now triggers this workflow. If tests fail, the build fails, and you see a red ✗ on your commit. If they pass, you get a green ✓.

For Node.js, the setup is nearly identical — just swap setup-python for setup-node and pytest for npm test.

Test Database Migrations and Schema Changes

One of the trickiest parts of AI-assisted development is keeping your database schema in sync with your code. A migration that works locally might fail in production if you have real data.

Test your migrations explicitly:

# test_migrations.py
import pytest
from alembic.config import Config
from alembic.script import ScriptDirectory
from alembic.runtime.migration import MigrationContext
from alembic.operations import Operations

def test_migrations_upgrade_and_downgrade(client):
    """Verify migrations can upgrade and downgrade without errors."""
    # Upgrade to latest
    alembic_cfg = Config('alembic.ini')
    command.upgrade(alembic_cfg, 'head')
    
    # Verify the schema is as expected
    inspector = inspect(db.engine)
    tables = inspector.get_table_names()
    assert 'users' in tables
    assert 'posts' in tables
    
    # Downgrade and upgrade again
    command.downgrade(alembic_cfg, '-1')
    command.upgrade(alembic_cfg, 'head')
    
    # Should still be valid
    assert inspector.get_table_names() == tables

This catches the scenario where a migration applies cleanly but fails when you try to roll it back.

Test Environment Variables and Configuration

AI-generated code often has hardcoded assumptions about environment variables. Make sure your tests verify that your app reads config correctly.

# test_config.py
import os
import pytest
from app import create_app

def test_app_uses_test_database(monkeypatch):
    """Verify app reads DATABASE_URL from environment."""
    monkeypatch.setenv('DATABASE_URL', 'sqlite:///:memory:')
    monkeypatch.setenv('SECRET_KEY', 'test-secret')
    
    app = create_app()
    assert app.config['SQLALCHEMY_DATABASE_URI'] == 'sqlite:///:memory:'
    assert app.config['SECRET_KEY'] == 'test-secret'

def test_app_fails_without_required_env(monkeypatch):
    """Verify app fails gracefully if required env vars are missing."""
    monkeypatch.delenv('SECRET_KEY', raising=False)
    
    with pytest.raises(ValueError, match='SECRET_KEY'):
        create_app()

Coverage — Know What You're Actually Testing

Code coverage tells you what percentage of your codebase is exercised by tests. It's not a perfect metric (you can have 100% coverage and still miss bugs), but it's a useful signal.

Python: pip install pytest-cov, then run pytest --cov=app.

Node.js: Jest includes coverage by default. Run npm test -- --coverage.

Aim for at least 70–80% coverage on critical paths (authentication, payments, data mutations). Don't obsess over 100%.

Testing on Your Linux Host

When you deploy to production on a Linux hosting environment (whether it's a managed VPS or a custom AWS setup like Vibesies provides), you want the same test suite to run before deployment.

Add a pre-deploy test step to your deployment script:

#!/bin/bash
# deploy.sh
set -e

echo "Running tests..."
pytest --cov=app

if [ $? -ne 0 ]; then
  echo "Tests failed. Aborting deploy."
  exit 1
fi

echo "Tests passed. Deploying..."
git pull origin main
pip install -r requirements.txt
alembic upgrade head
sudo systemctl restart myapp
echo "Deploy complete."

This ensures you never push broken code to production, even if you accidentally skip the CI check locally.

Common Pitfalls and How to Avoid Them

• Tests that pass locally but fail in CI: Usually due to missing environment variables or database state. Use fixtures to set up a clean state before each test.
• Tests that are too slow: Slow tests get skipped. Use in-memory databases for unit/integration tests, and only test against real databases in a separate "smoke test" suite.
• Flaky tests that sometimes pass, sometimes fail: Often caused by timing issues (e.g., waiting for async operations). Use explicit waits and mocking instead of arbitrary sleep() calls.
• Tests that don't actually test anything: If your test just calls a function without asserting the result, it's not a test. Claude Code sometimes does this — review generated tests and add assertions where they're missing.

Putting It All Together: A Testing Checklist

• ☐ Write unit tests for business logic (calculations, validations, data transformations).
• ☐ Write integration tests for API endpoints and database queries.
• ☐ Set up a CI pipeline (GitHub Actions, GitLab CI, or similar) to run tests on every push.
• ☐ Test database migrations explicitly, including rollbacks.
• ☐ Verify environment variables are read correctly and required vars are enforced.
• ☐ Measure code coverage and aim for 70–80% on critical paths.
• ☐ Add a pre-deploy test step to your deployment script.
• ☐ Review AI-generated tests — they're a starting point, not the final word.

Conclusion: Testing Gives You Speed and Confidence

Automated testing for AI-built sites on Linux hosting isn't extra work — it's the thing that lets you move fast without breaking things. Unit tests, integration tests, and CI pipelines catch bugs before they reach users, and they free you to refactor and improve without fear.

If you're running a site on a managed Linux VPS or custom hosting setup, automated testing is your safety net. Set it up early, keep it simple, and let your AI pair programmer help you write the tests. The time you invest now pays back tenfold when you deploy with confidence.