234 lines
8.2 KiB
Markdown
234 lines
8.2 KiB
Markdown
# ShortLink — URL Shortener with Analytics
|
||
|
||
## Description
|
||
|
||
ShortLink is a self-hosted URL shortening service with a real-time analytics dashboard.
|
||
Users paste a long URL, receive a short link (e.g. `https://yourdomain.com/s/abc123`),
|
||
and can track how many times the link was visited, when, and from where.
|
||
The application targets teams and developers who need a private, self-controlled link shortener
|
||
without relying on third-party services like bit.ly.
|
||
|
||
---
|
||
|
||
## Cloud Infrastructure
|
||
|
||
**Cloud provider:** azure
|
||
|
||
| Component | GCP Service / Tool | Purpose |
|
||
|---|---|---|
|
||
| Virtual machine | Compute Engine e2-small | Hosts all containers |
|
||
| Static IP | Compute Engine – External IP | Fixed address for DNS |
|
||
| Firewall | VPC Firewall rules | Allow HTTP (80) and HTTPS (443) |
|
||
| DNS | Any registrar → A record | Maps domain to the VM IP |
|
||
| SSL certificate | Let's Encrypt via Certbot | Free automatic HTTPS certificate |
|
||
| Container runtime | Docker Engine | Runs all application containers |
|
||
| Orchestration | Docker Compose | Manages multi-container lifecycle |
|
||
|
||
**Docker objects:**
|
||
|
||
| Container | Image | Role |
|
||
|---|---|---|
|
||
| `shortlink_nginx` | `nginx:alpine` | Reverse proxy, SSL termination, serves static frontend |
|
||
| `shortlink_backend` | Built from `./backend/Dockerfile` | FastAPI REST API, business logic |
|
||
| `shortlink_db` | `postgres:15-alpine` | Relational database (links + visits tables) |
|
||
|
||
**Named volumes:**
|
||
|
||
| Volume | Mounted in | Contents |
|
||
|---|---|---|
|
||
| `postgres_data` | `/var/lib/postgresql/data` | All database data (persistent) |
|
||
| `nginx_logs` | `/var/log/nginx` | Access and error logs |
|
||
|
||
**Network:** All three containers share a Docker bridge network (`app_network`) so they can reach each other by container name (e.g. `backend:8000`, `db:5432`).
|
||
|
||
---
|
||
|
||
## Cost Analysis (1 000 daily users, 50 GB data)
|
||
|
||
Estimated traffic: ~1 000 requests/day → ~30 000/month (light load).
|
||
|
||
| Resource | Specification | Monthly price | Annual price |
|
||
|---|---|---|---|
|
||
| Compute Engine e2-small | 2 vCPU, 2 GB RAM, region us-central1 | $13.60 | $163 |
|
||
| Persistent disk (SSD boot) | 30 GB | $5.10 | $61 |
|
||
| Additional SSD (database) | 50 GB | $8.50 | $102 |
|
||
| External IP address (static) | 1 address | $7.30 | $88 |
|
||
| Egress traffic | ~50 GB/month outbound | $4.50 | $54 |
|
||
| SSL certificate | Let's Encrypt | $0 | $0 |
|
||
| **Total** | | **~$39** | **~$468** |
|
||
|
||
> Prices based on GCP us-central1 on-demand pricing (2024). Costs can be reduced ~37% by using a 1-year committed-use discount.
|
||
> Snapshot for backup: ~$0.026/GB/month × 50 GB = $1.30/month extra.
|
||
|
||
---
|
||
|
||
## Files
|
||
|
||
```
|
||
sk1/
|
||
├── prepare-app.sh # Deploy: installs Docker, Certbot, gets SSL cert, starts app
|
||
├── remove-app.sh # Teardown: stops and removes all containers and volumes
|
||
├── docker-compose.yml # Defines all three services, volumes and network
|
||
├── .env.example # Template for required environment variables (copy to .env)
|
||
├── README.md # This file
|
||
│
|
||
├── backend/
|
||
│ ├── Dockerfile # Builds Python 3.11-slim image with FastAPI
|
||
│ ├── requirements.txt # Python dependencies (fastapi, uvicorn, psycopg2)
|
||
│ └── main.py # REST API: POST /api/shorten, GET /api/stats,
|
||
│ # DELETE /api/links/{code}, GET /s/{code}
|
||
│
|
||
├── init-db/
|
||
│ └── init.sql # Creates tables (links, visits) and indexes on first run
|
||
│
|
||
├── nginx/
|
||
│ ├── nginx.conf.template # Nginx config with DOMAIN_PLACEHOLDER (filled by prepare-app.sh)
|
||
│ └── html/
|
||
│ └── index.html # Single-page frontend (vanilla JS, dark theme)
|
||
│
|
||
└── scripts/
|
||
└── backup.sh # Dumps the database to a gzip file, keeps last 7
|
||
```
|
||
|
||
---
|
||
|
||
## Configuration
|
||
|
||
All runtime secrets and settings live in `.env` (never committed to Git).
|
||
|
||
| Variable | Description |
|
||
|---|---|
|
||
| `DB_NAME` | PostgreSQL database name |
|
||
| `DB_USER` | PostgreSQL user |
|
||
| `DB_PASSWORD` | PostgreSQL password (secret) |
|
||
| `BASE_URL` | Public URL shown in short links (e.g. `https://yourdomain.com`) |
|
||
| `SECRET_KEY` | Application secret for future JWT use (secret) |
|
||
| `ADMIN_TOKEN` | Bearer token required to delete links via API (secret) |
|
||
| `DOMAIN` | Domain name used by Certbot and nginx (e.g. `yourdomain.com`) |
|
||
| `EMAIL` | Email for Let's Encrypt certificate notifications |
|
||
| `BACKUP_DIR` | Where backups are written (default: `./backups`) |
|
||
|
||
`docker-compose.yml` reads these via `${VAR}` substitution and passes them as container environment variables. No secrets appear in any source file or in Git.
|
||
|
||
---
|
||
|
||
## Deployment Instructions
|
||
|
||
### Prerequisites
|
||
|
||
- A GCP Compute Engine VM running Ubuntu 22.04 LTS (e2-small or larger)
|
||
- An external static IP assigned to the VM
|
||
- Firewall rules allowing TCP 80 and TCP 443
|
||
- A domain name with an A record pointing to the VM IP
|
||
- SSH access to the VM
|
||
|
||
### Steps
|
||
|
||
```bash
|
||
# 1. SSH into the VM
|
||
gcloud compute ssh <INSTANCE_NAME> --zone <ZONE>
|
||
|
||
# 2. Clone or copy the project to the VM
|
||
git clone <YOUR_GIT_REPO_URL>
|
||
cd sk1
|
||
|
||
# 3. Create and fill in the environment file
|
||
cp .env.example .env
|
||
nano .env # fill in all values
|
||
|
||
# 4. Make scripts executable
|
||
chmod +x prepare-app.sh remove-app.sh scripts/backup.sh
|
||
|
||
# 5. Run the deployment script
|
||
./prepare-app.sh
|
||
|
||
# Done — visit https://<DOMAIN>
|
||
```
|
||
|
||
### Using the application
|
||
|
||
1. Open `https://<DOMAIN>` in a web browser.
|
||
2. Paste any URL into the input field and click **Shorten**.
|
||
3. Copy the short link and share it — every click is tracked.
|
||
4. The **Analytics** table below shows all links with visit counts.
|
||
5. To use a custom short code, click **⚙ Custom code** before shortening.
|
||
6. To delete a link, click **Del** in the table and enter the `ADMIN_TOKEN`.
|
||
|
||
---
|
||
|
||
## Backup
|
||
|
||
```bash
|
||
# Run the backup script (from the sk1 directory)
|
||
./scripts/backup.sh
|
||
|
||
# Backups are saved as ./backups/shortlink_YYYYMMDD_HHMMSS.sql.gz
|
||
# The last 7 backups are kept automatically.
|
||
|
||
# To restore:
|
||
gunzip -c backups/shortlink_<timestamp>.sql.gz \
|
||
| docker exec -i shortlink_db psql -U "$DB_USER" -d "$DB_NAME"
|
||
```
|
||
|
||
---
|
||
|
||
## Viewing Access Logs
|
||
|
||
```bash
|
||
# Live nginx access log (HTTP requests from the internet)
|
||
docker exec shortlink_nginx tail -f /var/log/nginx/access.log
|
||
|
||
# Or from the named volume on the host
|
||
docker run --rm -v shortlink_nginx_logs:/logs alpine tail -f /logs/access.log
|
||
|
||
# Last 100 lines of access log
|
||
docker exec shortlink_nginx tail -100 /var/log/nginx/access.log
|
||
|
||
# Error log
|
||
docker exec shortlink_nginx tail -f /var/log/nginx/error.log
|
||
|
||
# All container logs (backend API logs including each redirect)
|
||
docker compose logs -f
|
||
```
|
||
|
||
---
|
||
|
||
## Stopping / Removing the Application
|
||
|
||
```bash
|
||
./remove-app.sh
|
||
```
|
||
|
||
This stops and removes containers, the Docker network, and named volumes (database is deleted). SSL certificates are preserved.
|
||
|
||
---
|
||
|
||
## Script Conditions
|
||
|
||
### prepare-app.sh
|
||
|
||
- Must be run on a **Ubuntu 22.04** server (GCP Compute Engine VM or equivalent)
|
||
- The VM must have **ports 80 and 443 open** in the firewall
|
||
- **DNS** must be configured: the domain's A record must point to the VM's external IP **before running the script** (Certbot validates this)
|
||
- `.env` file must exist with all required variables filled in
|
||
- Internet access required (to pull Docker images, install packages, contact Let's Encrypt)
|
||
- The script is idempotent: safe to run again if interrupted
|
||
|
||
### remove-app.sh
|
||
|
||
- Must be run from the `sk1` directory on the same VM
|
||
- Docker and Docker Compose must be installed
|
||
|
||
---
|
||
|
||
## External Resources
|
||
|
||
| Resource | Type | Usage |
|
||
|---|---|---|
|
||
| FastAPI documentation (fastapi.tiangolo.com) | Official docs | API routing, response models, middleware |
|
||
| Docker Compose file reference (docs.docker.com) | Official docs | Service configuration, health checks, volumes |
|
||
| Certbot documentation (certbot.eff.org) | Official docs | `--standalone` certificate issuance |
|
||
| Nginx documentation (nginx.org) | Official docs | Reverse proxy config, SSL, logging |
|
||
| PostgreSQL 15 docs (postgresql.org) | Official docs | SQL schema, pg_dump backup |
|
||
| **gwen ** | Generative AI | Used for: generating boilerplate FastAPI route stubs, suggesting nginx proxy_pass configuration, explaining Let's Encrypt certbot flags, reviewing shell script error handling.. |
|