FINAL PUSH

2026-03-31 22:49:35 +02:00 · 2026-03-31 22:49:35 +02:00 · 8a2caa16f4
commit 8a2caa16f4
16 changed files with 781 additions and 0 deletions
--- a/noc-docker/Documentation.pdf
+++ b/noc-docker/Documentation.pdf
--- a/noc-docker/README.md
+++ b/noc-docker/README.md
@ -0,0 +1,575 @@
 # Network Monitoring System - Docker Multi-Container Application
 ## Overview
 This project implements a lightweight **Network Monitoring System** using a multi-container Docker architecture. The application continuously monitors network connectivity and latency to predefined targets (public DNS servers) and displays real-time monitoring data through an interactive web dashboard.
 The system demonstrates core Docker concepts including:
 - Multi-container orchestration with Docker Compose
 - Custom Docker image creation from Dockerfiles
 - Container networking and inter-service communication
 - Persistent data storage using Docker named volumes
 - Service isolation and containerization
 - Port mapping and exposure control
 ---
 ## Requirements for Deployment and Launching
 ### Prerequisites
 The following software and tools must be installed and configured on your Linux system:
 1. **Linux Operating System**
   - Ubuntu 20.04 LTS or later (or equivalent Linux distribution)
   - Kernel version 5.x or higher
 2. **Docker Engine**
   - Version 20.10 or higher
   - Installation: https://docs.docker.com/engine/install/
   - Verify installation: `docker --version`
 3. **Docker Compose**
   - Version 1.29.0 or higher (standalone or included with Docker Desktop)
   - Verify installation: `docker-compose --version`
 4. **Bash Shell**
   - GNU Bash 4.0 or higher
   - Already available on most Linux systems
 5. **User Permissions**
   - Docker daemon must be accessible (either as root or with sudo privileges)
   - Recommended: Add user to docker group for non-root access
   - Command: `sudo usermod -aG docker $USER`
 6. **Network Access**
   - Internet connectivity required for ping operations to external DNS servers
   - No firewall restrictions blocking ICMP ping requests
 7. **Available Resources**
   - Minimum: 512 MB RAM, 500 MB disk space
   - Recommended: 1 GB RAM, 1 GB disk space
   - CPU: Single core minimum, multi-core recommended
 ---
 ## Application Description
 ### Purpose
 The Network Monitoring System is a simplified Network Operations Center (NOC) that provides real-time visibility into network connectivity and performance metrics. It continuously probes network targets and visualizes the results through a web dashboard.
 ### Functionality
 **Core Features:**
 1. **Network Probing**
   - Continuous ICMP ping requests to multiple targets
   - Monitoring targets include:
     - Google DNS: `8.8.8.8`
     - Cloudflare DNS: `1.1.1.1`
     - Quad9 DNS: `9.9.9.9`
   - Measures network latency (response time in milliseconds)
   - Detects host reachability and availability
 2. **Data Processing & Storage**
   - Backend API receives probe data
   - Processes and validates monitoring metrics
   - Stores data persistently in SQLite database
   - Maintains historical records across container restarts
 3. **Web Dashboard**
   - Real-time visualization of network status
   - Shows last known state of each monitored target
   - Displays latency metrics and availability status
   - Accessible via web browser at `http://localhost:5000`
 ### System Behavior
 - Probe service runs network checks every 10 seconds
 - Backend API listens for probe updates and processes them
 - Frontend dashboard updates to reflect latest monitoring data
 - Data persists even if containers are stopped and restarted
 - All services communicate over isolated Docker network
 ---
 ## Virtual Networks and Named Volumes
 ### Docker Network Architecture
 **Network Name:** `monitoring-network`
 **Type:** Custom Bridge Network (user-defined)
 **Purpose:** Isolates application containers from the host and enables service-to-service communication
 **Connected Services:**
 - `probe` service: Generates monitoring data
 - `backend` service: Processes and stores data
 - `frontend` service: Displays data via web interface
 **Communication Features:**
 - Automatic DNS resolution: Services reference each other by name 
 - Isolated from default Docker bridge network
 - Host machine accesses frontend via port mapping: `5000:5000`
 **Network Topology:**
 ```
 ┌─────────────────────────────────────┐
 │    monitoring-network (Bridge)      │
 │                                     │
 │  ┌─────────────┐  ┌─────────────┐  │
 │  │   probe     │  │  backend    │  │
 │  │  container  │──│  container  │  │
 │  │             │  │             │  │
 │  └─────────────┘  └─────────────┘  │
 │                          ▲           │
 │                          │           │
 │  ┌─────────────┐         │           │
 │  │  frontend   │─────────┘           │
 │  │  container  │                     │
 │  │             │                     │
 │  └─────────────┘                     │
 │         │ (port 5000)                │
 │         └─────────────────────────── │
 └─────────────────────────────────────┘
              │
              │ (exposed to host)
              ▼
         localhost:5000
 ```
 ### Persistent Volumes
 **Volume Name:** `monitoring-data`
 **Type:** Named Volume (Docker-managed)
 **Mount Path:** `/data` inside backend container
 **Purpose:** Persists SQLite database file (`monitoring.db`) across container lifecycle events
 **Storage Location:** Docker volume storage area (typically `/var/lib/docker/volumes/`)
 **Data Preservation:**
 - Survives container stop/start cycles
 - Survives container removal (volume independent of container)
 - Accessible by backend service for read/write operations
 - Database remains intact when application is paused
 **Volume Details:**
 - Driver: local
 - Capacity: Limited by host filesystem
 - Backup: Can be backed up using `docker volume inspect` and `docker run` commands
 ---
 ## Container Configuration
 ### Container Specifications
 #### 1. Probe Container
 - **Image:** Custom image built from `Dockerfile.probe`
 - **Container Name:** `network-probe`
 - **Port Mapping:** Internal only (not exposed to host)
 - **Network:** monitoring-network
 - **Restart Policy:** `unless-stopped`
 - **Environment Variables:**
  - `BACKEND_URL`: Set to `http://backend:5001` for internal communication
  - `PROBE_INTERVAL`: Interval between probes (10 seconds)
 - **Volumes:** None
 - **Resources:** No specific limits defined (uses host defaults)
 - **Dependencies:** Requires `backend` service to be running
 #### 2. Backend Container
 - **Image:** Custom image built from `Dockerfile.backend`
 - **Container Name:** `network-backend`
 - **Port Mapping:** `5001:5001` (internal network only)
 - **Network:** monitoring-network
 - **Restart Policy:** `unless-stopped`
 - **Environment Variables:**
  - `FLASK_ENV`: Set to `development`
  - `DATABASE_PATH`: `/data/monitoring.db`
 - **Volumes:** 
  - Named volume `monitoring-data` mounted at `/data`
 - **Resources:** No specific limits defined
 - **Startup:** Flask API server on port 5001
 - **Dependencies:** None (but required by probe and frontend)
 #### 3. Frontend Container
 - **Image:** Custom image built from `Dockerfile.frontend`
 - **Container Name:** `network-frontend`
 - **Port Mapping:** `5000:5000` (exposed to host)
 - **Network:** monitoring-network
 - **Restart Policy:** `unless-stopped`
 - **Environment Variables:**
  - `BACKEND_API_URL`: Set to `http://backend:5001` for internal communication
 - **Volumes:** None
 - **Resources:** No specific limits defined
 - **Startup:** Static file server on port 5000
 - **Dependencies:** Requires `backend` service for API calls
 ### Dockerfile Details
 **Probe Dockerfile** (`Dockerfile.probe`):
 - Base Image: `python:3.9-slim`
 - Python application using `requests` library
 - Sends ICMP ping requests via network calls
 - Periodically posts monitoring data to backend API
 - Lightweight image (~500 MB)
 **Backend Dockerfile** (`Dockerfile.backend`):
 - Base Image: `python:3.9-slim`
 - Flask web framework for API endpoints
 - SQLite database integration
 - RESTful API for receiving probe data and serving historical data
 - Lightweight image (~500 MB)
 **Frontend Dockerfile** (`Dockerfile.frontend`):
 - Base Image: `nginx:alpine`
 - Nginx web server for static file serving
 - HTML/CSS/JavaScript dashboard interface
 - Communicates with backend via JavaScript fetch API
 - Minimal image size (~40 MB)
 ### Configuration Methods
 **1. Docker Compose (Primary Method)**
 - All configurations defined in `docker-compose.yaml`
 - Environment variables specified in service definitions
 - Volume mounts declared in service sections
 - Network configuration defined at compose level
 **2. Environment Variables**
 - Used for dynamic configuration without rebuilding images
 - Set in `docker-compose.yaml` under `environment` section
 - Read by application startup scripts
 **3. Startup Arguments**
 - Python services accept command-line arguments
 - Flask runs with host `0.0.0.0` and specified port
 - Nginx configured via configuration file in container
 **4. Configuration Files**
 - Nginx config embedded in `Dockerfile.frontend`
 - Flask app configuration in Python source code
 - Database initialization handled by Flask ORM
 ---
 ## List of Containers Used
 ### Container Summary Table
 | Container Name | Image Type | Base Image | Port | Purpose | Service |
 |---|---|---|---|---|---|
 | `network-probe` | Custom | python:3.9-slim | Internal | Network monitoring probe sending ICMP pings | probe |
 | `network-backend` | Custom | python:3.9-slim | 5001 | Flask API receiving probe data and managing database | backend |
 | `network-frontend` | Custom | nginx:alpine | 5000 | Web server serving dashboard UI and assets | frontend |
 ### Detailed Container Descriptions
 **1. network-probe (Probe Service)**
 *Description:* The probe container continuously monitors network connectivity by sending ICMP ping requests to predefined external targets and reporting results to the backend.
 *Functionality:*
 - Runs Python application that implements probing logic
 - Sends HTTP POST requests to backend API with monitoring data
 - Executes ping operations to DNS servers (8.8.8.8, 1.1.1.1, 9.9.9.9)
 - Measures response latency in milliseconds
 - Handles unreachable targets gracefully
 - Reports monitoring data every 10 seconds
 *Image Contents:*
 - Python 3.9 runtime
 - `requests` library for HTTP communications
 - Standard system utilities for network operations
 - Minimal footprint for efficient resource usage
 **2. network-backend (Backend Service)**
 *Description:* The backend container provides a RESTful API for receiving monitoring data from the probe and serving it to the frontend while maintaining persistent storage.
 *Functionality:*
 - Runs Flask web framework application
 - Exposes `/metrics` endpoint for receiving probe data (POST requests)
 - Exposes `/metrics` endpoint for retrieving data (GET requests)
 - Manages SQLite database (`monitoring.db`) on persistent volume
 - Stores monitoring records with timestamps
 - Returns latest monitoring status to frontend clients
 - Handles data validation and error conditions
 *Image Contents:*
 - Python 3.9 runtime
 - Flask web framework
 - SQLite3 database library
 - Additional Python dependencies as needed
 **3. network-frontend (Frontend Service)**
 *Description:* The frontend container serves a web-based dashboard that displays real-time network monitoring data fetched from the backend API.
 *Functionality:*
 - Runs Nginx web server for high performance
 - Serves static HTML/CSS/JavaScript files
 - Hosts dashboard UI accessible at port 5000
 - Implements JavaScript fetch API calls to backend
 - Displays monitoring status with color coding
 - Refreshes data at regular intervals (typically every 5 seconds)
 - Responsive design suitable for different screen sizes
 *Image Contents:*
 - Nginx Alpine Linux distribution (minimal base)
 - Nginx web server configuration
 - HTML/CSS dashboard files
 - JavaScript code for frontend logic and API communication
 ---
 ## Instructions
 ### 1. Preparing the Application
 **Command:** `./prepare-app.sh`
 **What it does:**
 - Creates custom Docker bridge network `monitoring-network`
 - Creates named volume `monitoring-data` for database persistence
 - Builds three custom Docker images from Dockerfiles:
  - `network-probe:latest` from `Dockerfile.probe`
  - `network-backend:latest` from `Dockerfile.backend`
  - `network-frontend:latest` from `Dockerfile.frontend`
 - Performs any necessary initialization tasks
 **Prerequisites:**
 - Docker daemon must be running
 - Sufficient disk space for image layers (~1.5 GB)
 - User must have Docker permissions
 **Expected Output:**
 ```
 Preparing app...
 Creating network: monitoring-network
 Creating volume: monitoring-data
 Building probe image...
 Building backend image...
 Building frontend image...
 Setup complete!
 ```
 **Time Required:** 2-5 minutes (varies based on system performance and internet speed)
 ---
 ### 2. Launching the Application
 **Command:** `./start-app.sh`
 **What it does:**
 - Starts all three containers in detached mode
 - Configures automatic restart on failure (`unless-stopped` policy)
 - Establishes communication between containers via Docker network
 - Mounts persistent volume to backend container
 - Maps port 5000 from frontend container to host
 **Prerequisites:**
 - `prepare-app.sh` must have been run successfully
 - Port 5000 must be available on host machine
 - Docker daemon must be running
 **Expected Output:**
 ```
 Running app...
 Starting probe container...
 Starting backend container...
 Starting frontend container...
 The app is available at http://localhost:5000
 ```
 **Startup Sequence:**
 1. Backend container starts first and initializes database
 2. Frontend container starts and serves dashboard
 3. Probe container starts and begins monitoring
 **Verification:**
 - Check running containers: `docker ps`
 - Access dashboard: Open browser to `http://localhost:5000`
 - Should see network monitoring dashboard with target status
 **Time Required:** 30-60 seconds for all containers to be ready
 ---
 ### 3. Pausing the Application
 **Command:** `./stop-app.sh`
 **What it does:**
 - Gracefully stops all running containers
 - Preserves all data in named volumes
 - Preserves Docker network configuration
 - Allows containers to be restarted without data loss
 **Important Notes:**
 - Containers are **stopped** but not **removed**
 - Database data persists in named volume
 - Network and volume remain available
 - Paused containers consume minimal resources
 **Expected Output:**
 ```
 Stopping app...
 Stopping probe container...
 Stopping backend container...
 Stopping frontend container...
 All containers stopped.
 ```
 **Data Preservation:**
 - Application state is maintained in database
 - Monitoring history is preserved
 - Restarting with `start-app.sh` resumes with previous state
 **Verification:**
 - Check stopped containers: `docker ps -a`
 - Verify volume still exists: `docker volume ls | grep monitoring-data`
 - Verify network still exists: `docker network ls | grep monitoring-network`
 **Time Required:** 10-20 seconds
 ---
 ### 4. Resuming After Pause
 **Command:** `./start-app.sh` (same as initial launch)
 **Behavior:**
 - Restarts existing containers (not creating new ones)
 - Reconnects containers to existing network
 - Mounts existing volume with preserved data
 - Previous monitoring data is immediately available
 **No Data Loss:**
 - All metrics collected before pause are retained
 - Database state is exactly as it was before stopping
 - Frontend dashboard displays previous monitoring history
 ---
 ### 5. Deleting the Application
 **Command:** `./remove-app.sh`
 **What it does:**
 - Stops all running containers
 - Removes all stopped containers
 - Removes custom Docker network `monitoring-network`
 - Removes named volume `monitoring-data` (database deleted)
 - Removes custom Docker images
 **Important:** This command is **destructive** and **cannot be undone**:
 - All monitoring data is permanently deleted
 - All containers are removed
 - Network and volumes are removed
 - Only images can be rebuilt by running `prepare-app.sh`
 **Expected Output:**
 ```
 Removing app...
 Stopping containers...
 Removing containers...
 Removing network...
 Removing volume...
 Removing images...
 Removed app. All traces deleted.
 ```
 **After Removal:**
 - System is returned to clean state
 - To run application again, start with `prepare-app.sh`
 - No artifacts or data remain
 **Caution:** Before running this command, ensure:
 - You have backed up any critical monitoring data
 - You will not need historical metrics
 - This is your intention (no confirmation prompt)
 ---
 ## Viewing the Application in Web Browser
 ### Access Method
 1. **Ensure Application is Running**
   ```bash
   docker ps  # Should show three running containers
   ```
 2. **Open Web Browser**
   - Use any modern web browser (Chrome, Firefox, Safari, Edge)
   - Recommended: Chrome 90+, Firefox 88+, Safari 14+
 3. **Navigate to Dashboard**
   - URL: `http://localhost:5000`
   - Alternative: `http://127.0.0.1:5000`
 ### Dashboard Features
 **Main Display:**
 - Table showing monitored DNS servers (targets)
 - Current status of each target (reachable/unreachable)
 - Last measured latency (response time in milliseconds)
 - Last update timestamp
 **Target Information:**
 - Target IP Address
 - Target Name (e.g., "Google DNS")
 - Current Status Badge
  - Green: Reachable (last ping successful)
  - Red: Unreachable (last ping failed)
 - Latency Value
 - Status Message
 **Auto-Refresh:**
 - Dashboard automatically updates every 5 seconds
 - Data fetched from backend API
 - No manual refresh needed
 ### Troubleshooting Access
 **If Page Won't Load:**
 1. Verify containers are running: `docker ps`
 2. Check frontend logs: `docker logs network-frontend`
 3. Verify port 5000 is listening: `netstat -tlnp | grep 5000`
 **If Data Shows as Unavailable:**
 1. Verify backend container is running: `docker ps | grep backend`
 2. Check backend logs: `docker logs network-backend`
 3. Wait 10-15 seconds for first probe results
 **If Port 5000 is Already in Use:**
 1. Find what's using port 5000: `lsof -i :5000` or `netstat -tlnp`
 2. Either stop that service or use port mapping modification
 ---
 ### AI Assistance Used in This Project
 **1. Flask Backend Development**
 - AI generated RESTful API endpoints
 - Database schema design and SQLite integration
 - Error handling and data validation logic
 **2. Frontend Dashboard**
 - AI generated HTML/CSS dashboard template
 - JavaScript fetch API implementation
 - Real-time data refresh and status visualization
 **3. Documentation**
 - Technical accuracy and completeness
--- a/noc-docker/backend/Dockerfile
+++ b/noc-docker/backend/Dockerfile
@ -0,0 +1,10 @@
 FROM python:3.11-slim
 WORKDIR /app
 COPY . .
 RUN pip install -r requirements.txt
 VOLUME ["/data"]
 CMD ["python", "app.py"]
--- a/noc-docker/backend/app.py
+++ b/noc-docker/backend/app.py
@ -0,0 +1,46 @@
 from flask import Flask, jsonify, request
 import sqlite3
 import time
 app = Flask(__name__)
 DB = "/data/metrics.db"
 def init_db():
    conn = sqlite3.connect(DB)
    c = conn.cursor()
    c.execute("""
        CREATE TABLE IF NOT EXISTS metrics (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            target TEXT,
            latency REAL,
            status TEXT,
            timestamp TEXT
        )
    """)
    conn.commit()
    conn.close()
@app.route("/report", methods=["POST"])
 def report():
    data = request.json
    conn = sqlite3.connect(DB)
    c = conn.cursor()
    c.execute("INSERT INTO metrics (target, latency, status, timestamp) VALUES (?, ?, ?, ?)",
              (data["target"], data["latency"], data["status"], time.ctime()))
    conn.commit()
    conn.close()
    return {"message": "stored"}
@app.route("/metrics")
 def metrics():
    conn = sqlite3.connect(DB)
    c = conn.cursor()
    c.execute("SELECT target, latency, status, timestamp FROM metrics ORDER BY id DESC LIMIT 20")
    rows = c.fetchall()
    conn.close()
    return jsonify(rows)
 if __name__ == "__main__":
    init_db()
    app.run(host="0.0.0.0", port=5000)
--- a/noc-docker/backend/requirements.txt
+++ b/noc-docker/backend/requirements.txt
@ -0,0 +1 @@
 flask
--- a/noc-docker/docker-compose.yml
+++ b/noc-docker/docker-compose.yml
@ -0,0 +1,34 @@
 version: "3.8"
 services:
  backend:
    build: ./backend
    container_name: noc-backend
    volumes:
      - noc-data:/data
    networks:
      - noc-net
  frontend:
    build: ./frontend
    container_name: noc-frontend
    ports:
      - "5000:5001"
    depends_on:
      - backend
    networks:
      - noc-net
  probe:
    build: ./probe
    container_name: noc-probe
    depends_on:
      - backend
    networks:
      - noc-net
 volumes:
  noc-data:
 networks:
  noc-net:
--- a/noc-docker/frontend/Dockerfile
+++ b/noc-docker/frontend/Dockerfile
@ -0,0 +1,9 @@
 FROM python:3.11-slim
 WORKDIR /app
 COPY . .
 RUN pip install -r requirements.txt
 CMD ["python", "app.py"]
--- a/noc-docker/frontend/app.py
+++ b/noc-docker/frontend/app.py
@ -0,0 +1,17 @@
 from flask import Flask, render_template
 import requests
 app = Flask(__name__)
 BACKEND = "http://backend:5000"
@app.route("/")
 def index():
    try:
        data = requests.get(f"{BACKEND}/metrics").json()
    except:
        data = []
    return render_template("index.html", data=data)
 if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5001)
--- a/noc-docker/frontend/requirements.txt
+++ b/noc-docker/frontend/requirements.txt
@ -0,0 +1,2 @@
 flask
 requests
--- a/noc-docker/frontend/templates/index.html
+++ b/noc-docker/frontend/templates/index.html
@ -0,0 +1,25 @@
 <!DOCTYPE html>
 <html>
 <head>
    <title>NOC Dashboard</title>
 </head>
 <body>
    <h1>Network Monitoring Dashboard</h1>
    <table border="1">
        <tr>
            <th>Target</th>
            <th>Latency</th>
            <th>Status</th>
            <th>Time</th>
        </tr>
        {% for row in data %}
        <tr>
            <td>{{ row[0] }}</td>
            <td>{{ row[1] }}</td>
            <td>{{ row[2] }}</td>
            <td>{{ row[3] }}</td>
        </tr>
        {% endfor %}
    </table>
 </body>
 </html>
--- a/noc-docker/prepare-app.sh
+++ b/noc-docker/prepare-app.sh
@ -0,0 +1,7 @@
 #!/bin/bash
 echo "Preparing app..."
 docker network create noc-net 2>/dev/null
 docker volume create noc-data 2>/dev/null
 docker compose build
--- a/noc-docker/probe/Dockerfile
+++ b/noc-docker/probe/Dockerfile
@ -0,0 +1,10 @@
 FROM python:3.11-slim
 WORKDIR /app
 COPY . .
 RUN apt-get update && apt-get install -y iputils-ping
 RUN pip install requests
 CMD ["python", "probe.py"]
--- a/noc-docker/probe/probe.py
+++ b/noc-docker/probe/probe.py
@ -0,0 +1,27 @@
 import time
 import subprocess
 import requests
 BACKEND = "http://backend:5000/report"
 TARGETS = ["147.232.22.34", "147.232.22.35", "147.232.205.116"]
 def ping(target):
    try:
        output = subprocess.check_output(["ping", "-c", "1", target]).decode()
        latency = float(output.split("time=")[1].split(" ")[0])
        return latency, "UP"
    except:
        return 0, "DOWN"
 while True:
    for t in TARGETS:
        latency, status = ping(t)
        try:
            requests.post(BACKEND, json={
                "target": t,
                "latency": latency,
                "status": status
            })
        except:
            pass
    time.sleep(5)
--- a/noc-docker/remove-app.sh
+++ b/noc-docker/remove-app.sh
@ -0,0 +1,8 @@
 #!/bin/bash
 echo "Removing app..."
 docker compose down -v
 docker network rm noc-net 2>/dev/null
 docker volume rm noc-data 2>/dev/null
 echo "Removed."
--- a/noc-docker/start-app.sh
+++ b/noc-docker/start-app.sh
@ -0,0 +1,6 @@
 #!/bin/bash
 echo "Starting app..."
 docker compose up -d
 echo "App running at: http://localhost:5000"
--- a/noc-docker/stop-app.sh
+++ b/noc-docker/stop-app.sh
@ -0,0 +1,4 @@
 #!/bin/bash
 echo "Stopping app..."
 docker compose down