commit 8a2caa16f480e0809f9f412353a306d1a29563a7 Author: Sarukesh Boominathan Date: Tue Mar 31 22:49:35 2026 +0200 FINAL PUSH diff --git a/noc-docker/Documentation.pdf b/noc-docker/Documentation.pdf new file mode 100644 index 0000000..b11c3f8 Binary files /dev/null and b/noc-docker/Documentation.pdf differ diff --git a/noc-docker/README.md b/noc-docker/README.md new file mode 100644 index 0000000..3b5f5f9 --- /dev/null +++ b/noc-docker/README.md @@ -0,0 +1,575 @@ +# Network Monitoring System - Docker Multi-Container Application + +## Overview + +This project implements a lightweight **Network Monitoring System** using a multi-container Docker architecture. The application continuously monitors network connectivity and latency to predefined targets (public DNS servers) and displays real-time monitoring data through an interactive web dashboard. + +The system demonstrates core Docker concepts including: +- Multi-container orchestration with Docker Compose +- Custom Docker image creation from Dockerfiles +- Container networking and inter-service communication +- Persistent data storage using Docker named volumes +- Service isolation and containerization +- Port mapping and exposure control + +--- + +## Requirements for Deployment and Launching + +### Prerequisites + +The following software and tools must be installed and configured on your Linux system: + +1. **Linux Operating System** + - Ubuntu 20.04 LTS or later (or equivalent Linux distribution) + - Kernel version 5.x or higher + +2. **Docker Engine** + - Version 20.10 or higher + - Installation: https://docs.docker.com/engine/install/ + - Verify installation: `docker --version` + +3. **Docker Compose** + - Version 1.29.0 or higher (standalone or included with Docker Desktop) + - Verify installation: `docker-compose --version` + +4. **Bash Shell** + - GNU Bash 4.0 or higher + - Already available on most Linux systems + +5. **User Permissions** + - Docker daemon must be accessible (either as root or with sudo privileges) + - Recommended: Add user to docker group for non-root access + - Command: `sudo usermod -aG docker $USER` + +6. **Network Access** + - Internet connectivity required for ping operations to external DNS servers + - No firewall restrictions blocking ICMP ping requests + +7. **Available Resources** + - Minimum: 512 MB RAM, 500 MB disk space + - Recommended: 1 GB RAM, 1 GB disk space + - CPU: Single core minimum, multi-core recommended + +--- + +## Application Description + +### Purpose + +The Network Monitoring System is a simplified Network Operations Center (NOC) that provides real-time visibility into network connectivity and performance metrics. It continuously probes network targets and visualizes the results through a web dashboard. + +### Functionality + +**Core Features:** + +1. **Network Probing** + - Continuous ICMP ping requests to multiple targets + - Monitoring targets include: + - Google DNS: `8.8.8.8` + - Cloudflare DNS: `1.1.1.1` + - Quad9 DNS: `9.9.9.9` + - Measures network latency (response time in milliseconds) + - Detects host reachability and availability + +2. **Data Processing & Storage** + - Backend API receives probe data + - Processes and validates monitoring metrics + - Stores data persistently in SQLite database + - Maintains historical records across container restarts + +3. **Web Dashboard** + - Real-time visualization of network status + - Shows last known state of each monitored target + - Displays latency metrics and availability status + - Accessible via web browser at `http://localhost:5000` + +### System Behavior + +- Probe service runs network checks every 10 seconds +- Backend API listens for probe updates and processes them +- Frontend dashboard updates to reflect latest monitoring data +- Data persists even if containers are stopped and restarted +- All services communicate over isolated Docker network + +--- + +## Virtual Networks and Named Volumes + +### Docker Network Architecture + +**Network Name:** `monitoring-network` + +**Type:** Custom Bridge Network (user-defined) + +**Purpose:** Isolates application containers from the host and enables service-to-service communication + +**Connected Services:** +- `probe` service: Generates monitoring data +- `backend` service: Processes and stores data +- `frontend` service: Displays data via web interface + +**Communication Features:** +- Automatic DNS resolution: Services reference each other by name +- Isolated from default Docker bridge network +- Host machine accesses frontend via port mapping: `5000:5000` + +**Network Topology:** + +``` +┌─────────────────────────────────────┐ +│ monitoring-network (Bridge) │ +│ │ +│ ┌─────────────┐ ┌─────────────┐ │ +│ │ probe │ │ backend │ │ +│ │ container │──│ container │ │ +│ │ │ │ │ │ +│ └─────────────┘ └─────────────┘ │ +│ ▲ │ +│ │ │ +│ ┌─────────────┐ │ │ +│ │ frontend │─────────┘ │ +│ │ container │ │ +│ │ │ │ +│ └─────────────┘ │ +│ │ (port 5000) │ +│ └─────────────────────────── │ +└─────────────────────────────────────┘ + │ + │ (exposed to host) + ▼ + localhost:5000 +``` + +### Persistent Volumes + +**Volume Name:** `monitoring-data` + +**Type:** Named Volume (Docker-managed) + +**Mount Path:** `/data` inside backend container + +**Purpose:** Persists SQLite database file (`monitoring.db`) across container lifecycle events + +**Storage Location:** Docker volume storage area (typically `/var/lib/docker/volumes/`) + +**Data Preservation:** +- Survives container stop/start cycles +- Survives container removal (volume independent of container) +- Accessible by backend service for read/write operations +- Database remains intact when application is paused + +**Volume Details:** +- Driver: local +- Capacity: Limited by host filesystem +- Backup: Can be backed up using `docker volume inspect` and `docker run` commands + +--- + +## Container Configuration + +### Container Specifications + +#### 1. Probe Container +- **Image:** Custom image built from `Dockerfile.probe` +- **Container Name:** `network-probe` +- **Port Mapping:** Internal only (not exposed to host) +- **Network:** monitoring-network +- **Restart Policy:** `unless-stopped` +- **Environment Variables:** + - `BACKEND_URL`: Set to `http://backend:5001` for internal communication + - `PROBE_INTERVAL`: Interval between probes (10 seconds) +- **Volumes:** None +- **Resources:** No specific limits defined (uses host defaults) +- **Dependencies:** Requires `backend` service to be running + +#### 2. Backend Container +- **Image:** Custom image built from `Dockerfile.backend` +- **Container Name:** `network-backend` +- **Port Mapping:** `5001:5001` (internal network only) +- **Network:** monitoring-network +- **Restart Policy:** `unless-stopped` +- **Environment Variables:** + - `FLASK_ENV`: Set to `development` + - `DATABASE_PATH`: `/data/monitoring.db` +- **Volumes:** + - Named volume `monitoring-data` mounted at `/data` +- **Resources:** No specific limits defined +- **Startup:** Flask API server on port 5001 +- **Dependencies:** None (but required by probe and frontend) + +#### 3. Frontend Container +- **Image:** Custom image built from `Dockerfile.frontend` +- **Container Name:** `network-frontend` +- **Port Mapping:** `5000:5000` (exposed to host) +- **Network:** monitoring-network +- **Restart Policy:** `unless-stopped` +- **Environment Variables:** + - `BACKEND_API_URL`: Set to `http://backend:5001` for internal communication +- **Volumes:** None +- **Resources:** No specific limits defined +- **Startup:** Static file server on port 5000 +- **Dependencies:** Requires `backend` service for API calls + +### Dockerfile Details + +**Probe Dockerfile** (`Dockerfile.probe`): +- Base Image: `python:3.9-slim` +- Python application using `requests` library +- Sends ICMP ping requests via network calls +- Periodically posts monitoring data to backend API +- Lightweight image (~500 MB) + +**Backend Dockerfile** (`Dockerfile.backend`): +- Base Image: `python:3.9-slim` +- Flask web framework for API endpoints +- SQLite database integration +- RESTful API for receiving probe data and serving historical data +- Lightweight image (~500 MB) + +**Frontend Dockerfile** (`Dockerfile.frontend`): +- Base Image: `nginx:alpine` +- Nginx web server for static file serving +- HTML/CSS/JavaScript dashboard interface +- Communicates with backend via JavaScript fetch API +- Minimal image size (~40 MB) + +### Configuration Methods + +**1. Docker Compose (Primary Method)** +- All configurations defined in `docker-compose.yaml` +- Environment variables specified in service definitions +- Volume mounts declared in service sections +- Network configuration defined at compose level + +**2. Environment Variables** +- Used for dynamic configuration without rebuilding images +- Set in `docker-compose.yaml` under `environment` section +- Read by application startup scripts + +**3. Startup Arguments** +- Python services accept command-line arguments +- Flask runs with host `0.0.0.0` and specified port +- Nginx configured via configuration file in container + +**4. Configuration Files** +- Nginx config embedded in `Dockerfile.frontend` +- Flask app configuration in Python source code +- Database initialization handled by Flask ORM + +--- + +## List of Containers Used + +### Container Summary Table + +| Container Name | Image Type | Base Image | Port | Purpose | Service | +|---|---|---|---|---|---| +| `network-probe` | Custom | python:3.9-slim | Internal | Network monitoring probe sending ICMP pings | probe | +| `network-backend` | Custom | python:3.9-slim | 5001 | Flask API receiving probe data and managing database | backend | +| `network-frontend` | Custom | nginx:alpine | 5000 | Web server serving dashboard UI and assets | frontend | + +### Detailed Container Descriptions + +**1. network-probe (Probe Service)** + +*Description:* The probe container continuously monitors network connectivity by sending ICMP ping requests to predefined external targets and reporting results to the backend. + +*Functionality:* +- Runs Python application that implements probing logic +- Sends HTTP POST requests to backend API with monitoring data +- Executes ping operations to DNS servers (8.8.8.8, 1.1.1.1, 9.9.9.9) +- Measures response latency in milliseconds +- Handles unreachable targets gracefully +- Reports monitoring data every 10 seconds + +*Image Contents:* +- Python 3.9 runtime +- `requests` library for HTTP communications +- Standard system utilities for network operations +- Minimal footprint for efficient resource usage + +**2. network-backend (Backend Service)** + +*Description:* The backend container provides a RESTful API for receiving monitoring data from the probe and serving it to the frontend while maintaining persistent storage. + +*Functionality:* +- Runs Flask web framework application +- Exposes `/metrics` endpoint for receiving probe data (POST requests) +- Exposes `/metrics` endpoint for retrieving data (GET requests) +- Manages SQLite database (`monitoring.db`) on persistent volume +- Stores monitoring records with timestamps +- Returns latest monitoring status to frontend clients +- Handles data validation and error conditions + +*Image Contents:* +- Python 3.9 runtime +- Flask web framework +- SQLite3 database library +- Additional Python dependencies as needed + +**3. network-frontend (Frontend Service)** + +*Description:* The frontend container serves a web-based dashboard that displays real-time network monitoring data fetched from the backend API. + +*Functionality:* +- Runs Nginx web server for high performance +- Serves static HTML/CSS/JavaScript files +- Hosts dashboard UI accessible at port 5000 +- Implements JavaScript fetch API calls to backend +- Displays monitoring status with color coding +- Refreshes data at regular intervals (typically every 5 seconds) +- Responsive design suitable for different screen sizes + +*Image Contents:* +- Nginx Alpine Linux distribution (minimal base) +- Nginx web server configuration +- HTML/CSS dashboard files +- JavaScript code for frontend logic and API communication + +--- + +## Instructions + +### 1. Preparing the Application + +**Command:** `./prepare-app.sh` + +**What it does:** +- Creates custom Docker bridge network `monitoring-network` +- Creates named volume `monitoring-data` for database persistence +- Builds three custom Docker images from Dockerfiles: + - `network-probe:latest` from `Dockerfile.probe` + - `network-backend:latest` from `Dockerfile.backend` + - `network-frontend:latest` from `Dockerfile.frontend` +- Performs any necessary initialization tasks + +**Prerequisites:** +- Docker daemon must be running +- Sufficient disk space for image layers (~1.5 GB) +- User must have Docker permissions + +**Expected Output:** +``` +Preparing app... +Creating network: monitoring-network +Creating volume: monitoring-data +Building probe image... +Building backend image... +Building frontend image... +Setup complete! +``` + +**Time Required:** 2-5 minutes (varies based on system performance and internet speed) + +--- + +### 2. Launching the Application + +**Command:** `./start-app.sh` + +**What it does:** +- Starts all three containers in detached mode +- Configures automatic restart on failure (`unless-stopped` policy) +- Establishes communication between containers via Docker network +- Mounts persistent volume to backend container +- Maps port 5000 from frontend container to host + +**Prerequisites:** +- `prepare-app.sh` must have been run successfully +- Port 5000 must be available on host machine +- Docker daemon must be running + +**Expected Output:** +``` +Running app... +Starting probe container... +Starting backend container... +Starting frontend container... +The app is available at http://localhost:5000 +``` + +**Startup Sequence:** +1. Backend container starts first and initializes database +2. Frontend container starts and serves dashboard +3. Probe container starts and begins monitoring + +**Verification:** +- Check running containers: `docker ps` +- Access dashboard: Open browser to `http://localhost:5000` +- Should see network monitoring dashboard with target status + +**Time Required:** 30-60 seconds for all containers to be ready + +--- + +### 3. Pausing the Application + +**Command:** `./stop-app.sh` + +**What it does:** +- Gracefully stops all running containers +- Preserves all data in named volumes +- Preserves Docker network configuration +- Allows containers to be restarted without data loss + +**Important Notes:** +- Containers are **stopped** but not **removed** +- Database data persists in named volume +- Network and volume remain available +- Paused containers consume minimal resources + +**Expected Output:** +``` +Stopping app... +Stopping probe container... +Stopping backend container... +Stopping frontend container... +All containers stopped. +``` + +**Data Preservation:** +- Application state is maintained in database +- Monitoring history is preserved +- Restarting with `start-app.sh` resumes with previous state + +**Verification:** +- Check stopped containers: `docker ps -a` +- Verify volume still exists: `docker volume ls | grep monitoring-data` +- Verify network still exists: `docker network ls | grep monitoring-network` + +**Time Required:** 10-20 seconds + +--- + +### 4. Resuming After Pause + +**Command:** `./start-app.sh` (same as initial launch) + +**Behavior:** +- Restarts existing containers (not creating new ones) +- Reconnects containers to existing network +- Mounts existing volume with preserved data +- Previous monitoring data is immediately available + +**No Data Loss:** +- All metrics collected before pause are retained +- Database state is exactly as it was before stopping +- Frontend dashboard displays previous monitoring history + +--- + +### 5. Deleting the Application + +**Command:** `./remove-app.sh` + +**What it does:** +- Stops all running containers +- Removes all stopped containers +- Removes custom Docker network `monitoring-network` +- Removes named volume `monitoring-data` (database deleted) +- Removes custom Docker images + +**Important:** This command is **destructive** and **cannot be undone**: +- All monitoring data is permanently deleted +- All containers are removed +- Network and volumes are removed +- Only images can be rebuilt by running `prepare-app.sh` + +**Expected Output:** +``` +Removing app... +Stopping containers... +Removing containers... +Removing network... +Removing volume... +Removing images... +Removed app. All traces deleted. +``` + +**After Removal:** +- System is returned to clean state +- To run application again, start with `prepare-app.sh` +- No artifacts or data remain + +**Caution:** Before running this command, ensure: +- You have backed up any critical monitoring data +- You will not need historical metrics +- This is your intention (no confirmation prompt) + +--- + +## Viewing the Application in Web Browser + +### Access Method + +1. **Ensure Application is Running** + ```bash + docker ps # Should show three running containers + ``` + +2. **Open Web Browser** + - Use any modern web browser (Chrome, Firefox, Safari, Edge) + - Recommended: Chrome 90+, Firefox 88+, Safari 14+ + +3. **Navigate to Dashboard** + - URL: `http://localhost:5000` + - Alternative: `http://127.0.0.1:5000` + +### Dashboard Features + +**Main Display:** +- Table showing monitored DNS servers (targets) +- Current status of each target (reachable/unreachable) +- Last measured latency (response time in milliseconds) +- Last update timestamp + +**Target Information:** +- Target IP Address +- Target Name (e.g., "Google DNS") +- Current Status Badge + - Green: Reachable (last ping successful) + - Red: Unreachable (last ping failed) +- Latency Value +- Status Message + +**Auto-Refresh:** +- Dashboard automatically updates every 5 seconds +- Data fetched from backend API +- No manual refresh needed + +### Troubleshooting Access + +**If Page Won't Load:** +1. Verify containers are running: `docker ps` +2. Check frontend logs: `docker logs network-frontend` +3. Verify port 5000 is listening: `netstat -tlnp | grep 5000` + +**If Data Shows as Unavailable:** +1. Verify backend container is running: `docker ps | grep backend` +2. Check backend logs: `docker logs network-backend` +3. Wait 10-15 seconds for first probe results + +**If Port 5000 is Already in Use:** +1. Find what's using port 5000: `lsof -i :5000` or `netstat -tlnp` +2. Either stop that service or use port mapping modification + +--- + +### AI Assistance Used in This Project + +**1. Flask Backend Development** +- AI generated RESTful API endpoints +- Database schema design and SQLite integration +- Error handling and data validation logic + +**2. Frontend Dashboard** +- AI generated HTML/CSS dashboard template +- JavaScript fetch API implementation +- Real-time data refresh and status visualization + +**3. Documentation** +- Technical accuracy and completeness + + + diff --git a/noc-docker/backend/Dockerfile b/noc-docker/backend/Dockerfile new file mode 100644 index 0000000..63e7c15 --- /dev/null +++ b/noc-docker/backend/Dockerfile @@ -0,0 +1,10 @@ +FROM python:3.11-slim + +WORKDIR /app +COPY . . + +RUN pip install -r requirements.txt + +VOLUME ["/data"] + +CMD ["python", "app.py"] diff --git a/noc-docker/backend/app.py b/noc-docker/backend/app.py new file mode 100644 index 0000000..5d999b4 --- /dev/null +++ b/noc-docker/backend/app.py @@ -0,0 +1,46 @@ +from flask import Flask, jsonify, request +import sqlite3 +import time + +app = Flask(__name__) + +DB = "/data/metrics.db" + +def init_db(): + conn = sqlite3.connect(DB) + c = conn.cursor() + c.execute(""" + CREATE TABLE IF NOT EXISTS metrics ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + target TEXT, + latency REAL, + status TEXT, + timestamp TEXT + ) + """) + conn.commit() + conn.close() + +@app.route("/report", methods=["POST"]) +def report(): + data = request.json + conn = sqlite3.connect(DB) + c = conn.cursor() + c.execute("INSERT INTO metrics (target, latency, status, timestamp) VALUES (?, ?, ?, ?)", + (data["target"], data["latency"], data["status"], time.ctime())) + conn.commit() + conn.close() + return {"message": "stored"} + +@app.route("/metrics") +def metrics(): + conn = sqlite3.connect(DB) + c = conn.cursor() + c.execute("SELECT target, latency, status, timestamp FROM metrics ORDER BY id DESC LIMIT 20") + rows = c.fetchall() + conn.close() + return jsonify(rows) + +if __name__ == "__main__": + init_db() + app.run(host="0.0.0.0", port=5000) diff --git a/noc-docker/backend/requirements.txt b/noc-docker/backend/requirements.txt new file mode 100644 index 0000000..7e10602 --- /dev/null +++ b/noc-docker/backend/requirements.txt @@ -0,0 +1 @@ +flask diff --git a/noc-docker/docker-compose.yml b/noc-docker/docker-compose.yml new file mode 100644 index 0000000..0584aab --- /dev/null +++ b/noc-docker/docker-compose.yml @@ -0,0 +1,34 @@ +version: "3.8" + +services: + backend: + build: ./backend + container_name: noc-backend + volumes: + - noc-data:/data + networks: + - noc-net + + frontend: + build: ./frontend + container_name: noc-frontend + ports: + - "5000:5001" + depends_on: + - backend + networks: + - noc-net + + probe: + build: ./probe + container_name: noc-probe + depends_on: + - backend + networks: + - noc-net + +volumes: + noc-data: + +networks: + noc-net: diff --git a/noc-docker/frontend/Dockerfile b/noc-docker/frontend/Dockerfile new file mode 100644 index 0000000..f0058be --- /dev/null +++ b/noc-docker/frontend/Dockerfile @@ -0,0 +1,9 @@ +FROM python:3.11-slim + +WORKDIR /app + +COPY . . + +RUN pip install -r requirements.txt + +CMD ["python", "app.py"] diff --git a/noc-docker/frontend/app.py b/noc-docker/frontend/app.py new file mode 100644 index 0000000..0f98047 --- /dev/null +++ b/noc-docker/frontend/app.py @@ -0,0 +1,17 @@ +from flask import Flask, render_template +import requests + +app = Flask(__name__) + +BACKEND = "http://backend:5000" + +@app.route("/") +def index(): + try: + data = requests.get(f"{BACKEND}/metrics").json() + except: + data = [] + return render_template("index.html", data=data) + +if __name__ == "__main__": + app.run(host="0.0.0.0", port=5001) diff --git a/noc-docker/frontend/requirements.txt b/noc-docker/frontend/requirements.txt new file mode 100644 index 0000000..30692b7 --- /dev/null +++ b/noc-docker/frontend/requirements.txt @@ -0,0 +1,2 @@ +flask +requests diff --git a/noc-docker/frontend/templates/index.html b/noc-docker/frontend/templates/index.html new file mode 100644 index 0000000..af7adfb --- /dev/null +++ b/noc-docker/frontend/templates/index.html @@ -0,0 +1,25 @@ + + + + NOC Dashboard + + +

Network Monitoring Dashboard

+ + + + + + + + {% for row in data %} + + + + + + + {% endfor %} +
TargetLatencyStatusTime
{{ row[0] }}{{ row[1] }}{{ row[2] }}{{ row[3] }}
+ + diff --git a/noc-docker/prepare-app.sh b/noc-docker/prepare-app.sh new file mode 100644 index 0000000..5ff81ee --- /dev/null +++ b/noc-docker/prepare-app.sh @@ -0,0 +1,7 @@ +#!/bin/bash +echo "Preparing app..." + +docker network create noc-net 2>/dev/null +docker volume create noc-data 2>/dev/null + +docker compose build diff --git a/noc-docker/probe/Dockerfile b/noc-docker/probe/Dockerfile new file mode 100644 index 0000000..94d4ef5 --- /dev/null +++ b/noc-docker/probe/Dockerfile @@ -0,0 +1,10 @@ +FROM python:3.11-slim + +WORKDIR /app + +COPY . . + +RUN apt-get update && apt-get install -y iputils-ping +RUN pip install requests + +CMD ["python", "probe.py"] diff --git a/noc-docker/probe/probe.py b/noc-docker/probe/probe.py new file mode 100644 index 0000000..0fdf3ad --- /dev/null +++ b/noc-docker/probe/probe.py @@ -0,0 +1,27 @@ +import time +import subprocess +import requests + +BACKEND = "http://backend:5000/report" +TARGETS = ["147.232.22.34", "147.232.22.35", "147.232.205.116"] + +def ping(target): + try: + output = subprocess.check_output(["ping", "-c", "1", target]).decode() + latency = float(output.split("time=")[1].split(" ")[0]) + return latency, "UP" + except: + return 0, "DOWN" + +while True: + for t in TARGETS: + latency, status = ping(t) + try: + requests.post(BACKEND, json={ + "target": t, + "latency": latency, + "status": status + }) + except: + pass + time.sleep(5) diff --git a/noc-docker/remove-app.sh b/noc-docker/remove-app.sh new file mode 100644 index 0000000..ff132cc --- /dev/null +++ b/noc-docker/remove-app.sh @@ -0,0 +1,8 @@ +#!/bin/bash +echo "Removing app..." + +docker compose down -v +docker network rm noc-net 2>/dev/null +docker volume rm noc-data 2>/dev/null + +echo "Removed." diff --git a/noc-docker/start-app.sh b/noc-docker/start-app.sh new file mode 100644 index 0000000..8054690 --- /dev/null +++ b/noc-docker/start-app.sh @@ -0,0 +1,6 @@ +#!/bin/bash +echo "Starting app..." + +docker compose up -d + +echo "App running at: http://localhost:5000" diff --git a/noc-docker/stop-app.sh b/noc-docker/stop-app.sh new file mode 100644 index 0000000..223e484 --- /dev/null +++ b/noc-docker/stop-app.sh @@ -0,0 +1,4 @@ +#!/bin/bash +echo "Stopping app..." + +docker compose down