ZKT26/noc-docker/README.md

# Network Monitoring System - Docker Multi-Container Application

## Overview

This project implements a lightweight **Network Monitoring System** using a multi-container Docker architecture. The application continuously monitors network connectivity and latency to predefined targets (public DNS servers) and displays real-time monitoring data through an interactive web dashboard.

The system demonstrates core Docker concepts including:
- Multi-container orchestration with Docker Compose
- Custom Docker image creation from Dockerfiles
- Container networking and inter-service communication
- Persistent data storage using Docker named volumes
- Service isolation and containerization
- Port mapping and exposure control

---

## Requirements for Deployment and Launching

### Prerequisites

The following software and tools must be installed and configured on your Linux system:

1. **Linux Operating System**
   - Ubuntu 20.04 LTS or later (or equivalent Linux distribution)
   - Kernel version 5.x or higher

2. **Docker Engine**
   - Version 20.10 or higher
   - Installation: https://docs.docker.com/engine/install/
   - Verify installation: `docker --version`

3. **Docker Compose**
   - Version 1.29.0 or higher (standalone or included with Docker Desktop)
   - Verify installation: `docker-compose --version`

4. **Bash Shell**
   - GNU Bash 4.0 or higher
   - Already available on most Linux systems

5. **User Permissions**
   - Docker daemon must be accessible (either as root or with sudo privileges)
   - Recommended: Add user to docker group for non-root access
   - Command: `sudo usermod -aG docker $USER`

6. **Network Access**
   - Internet connectivity required for ping operations to external DNS servers
   - No firewall restrictions blocking ICMP ping requests

7. **Available Resources**
   - Minimum: 512 MB RAM, 500 MB disk space
   - Recommended: 1 GB RAM, 1 GB disk space
   - CPU: Single core minimum, multi-core recommended

---

## Application Description

### Purpose

The Network Monitoring System is a simplified Network Operations Center (NOC) that provides real-time visibility into network connectivity and performance metrics. It continuously probes network targets and visualizes the results through a web dashboard.

### Functionality

**Core Features:**

1. **Network Probing**
   - Continuous ICMP ping requests to multiple targets
   - Monitoring targets include:
     - Google DNS: `8.8.8.8`
     - Cloudflare DNS: `1.1.1.1`
     - Quad9 DNS: `9.9.9.9`
   - Measures network latency (response time in milliseconds)
   - Detects host reachability and availability

2. **Data Processing & Storage**
   - Backend API receives probe data
   - Processes and validates monitoring metrics
   - Stores data persistently in SQLite database
   - Maintains historical records across container restarts

3. **Web Dashboard**
   - Real-time visualization of network status
   - Shows last known state of each monitored target
   - Displays latency metrics and availability status
   - Accessible via web browser at `http://localhost:5000`

### System Behavior

- Probe service runs network checks every 10 seconds
- Backend API listens for probe updates and processes them
- Frontend dashboard updates to reflect latest monitoring data
- Data persists even if containers are stopped and restarted
- All services communicate over isolated Docker network

---

## Virtual Networks and Named Volumes

### Docker Network Architecture

**Network Name:** `noc-net`

**Type:** Custom Bridge Network (user-defined)

**Purpose:** Isolates application containers from the host and enables service-to-service communication

**Connected Services:**
- `probe` service: Generates monitoring data
- `backend` service: Processes and stores data
- `frontend` service: Displays data via web interface

**Communication Features:**
- Automatic DNS resolution: Services reference each other by name 
- Isolated from default Docker bridge network
- Host machine accesses frontend via port mapping: `5000:5000`

**Network Topology:**

```
┌─────────────────────────────────────┐
│    monitoring-network (Bridge)      │
│                                     │
│  ┌─────────────┐  ┌─────────────┐  │
│  │   probe     │  │  backend    │  │
│  │  container  │──│  container  │  │
│  │             │  │             │  │
│  └─────────────┘  └─────────────┘  │
│                          ▲           │
│                          │           │
│  ┌─────────────┐         │           │
│  │  frontend   │─────────┘           │
│  │  container  │                     │
│  │             │                     │
│  └─────────────┘                     │
│         │ (port 5000)                │
│         └─────────────────────────── │
└─────────────────────────────────────┘
              │
              │ (exposed to host)
              ▼
         localhost:5000
```

### Persistent Volumes

**Volume Name:** `noc-data`

**Type:** Named Volume (Docker-managed)

**Mount Path:** `/data` inside backend container

**Purpose:** Persists SQLite database file across container lifecycle events

**Data Preservation:**
- Survives container stop/start cycles
- Survives container removal (volume independent of container)
- Accessible by backend service for read/write operations
- Database remains intact when application is paused

**Volume Details:**
- Driver: local
- Capacity: Limited by host filesystem
- Backup: Can be backed up using `docker volume inspect` and `docker run` commands

---

## Container Configuration

### Container Specifications

#### 1. Probe Container
- **Image:** Custom image built from `Dockerfile.probe`
- **Container Name:** `network-probe`
- **Port Mapping:** Internal only (not exposed to host)
- **Network:** monitoring-network
- **Restart Policy:** `unless-stopped`
- **Environment Variables:**
  - `BACKEND_URL`: Set to `http://backend:5001` for internal communication
  - `PROBE_INTERVAL`: Interval between probes (10 seconds)
- **Volumes:** None
- **Resources:** No specific limits defined (uses host defaults)
- **Dependencies:** Requires `backend` service to be running

#### 2. Backend Container
- **Image:** Custom image built from `Dockerfile.backend`
- **Container Name:** `network-backend`
- **Port Mapping:** `5001:5001` (internal network only)
- **Network:** monitoring-network
- **Restart Policy:** `unless-stopped`
- **Environment Variables:**
  - `FLASK_ENV`: Set to `development`
  - `DATABASE_PATH`: `/data/monitoring.db`
- **Volumes:** 
  - Named volume `monitoring-data` mounted at `/data`
- **Resources:** No specific limits defined
- **Startup:** Flask API server on port 5001
- **Dependencies:** None (but required by probe and frontend)

#### 3. Frontend Container
- **Image:** Custom image built from `Dockerfile.frontend`
- **Container Name:** `network-frontend`
- **Port Mapping:** `5000:5000` (exposed to host)
- **Network:** monitoring-network
- **Restart Policy:** `unless-stopped`
- **Environment Variables:**
  - `BACKEND_API_URL`: Set to `http://backend:5001` for internal communication
- **Volumes:** None
- **Resources:** No specific limits defined
- **Startup:** Static file server on port 5000
- **Dependencies:** Requires `backend` service for API calls

### Dockerfile Details

**Probe Dockerfile** (`Dockerfile.probe`):
- Base Image: `python:3.9-slim`
- Python application using `requests` library
- Sends ICMP ping requests via network calls
- Periodically posts monitoring data to backend API
- Lightweight image (~500 MB)

**Backend Dockerfile** (`Dockerfile.backend`):
- Base Image: `python:3.9-slim`
- Flask web framework for API endpoints
- SQLite database integration
- RESTful API for receiving probe data and serving historical data
- Lightweight image (~500 MB)

**Frontend Dockerfile** (`Dockerfile.frontend`):
- Base Image: `nginx:alpine`
- Nginx web server for static file serving
- HTML/CSS/JavaScript dashboard interface
- Communicates with backend via JavaScript fetch API
- Minimal image size (~40 MB)

### Configuration Methods

**1. Docker Compose (Primary Method)**
- All configurations defined in `docker-compose.yaml`
- Environment variables specified in service definitions
- Volume mounts declared in service sections
- Network configuration defined at compose level

**2. Environment Variables**
- Used for dynamic configuration without rebuilding images
- Set in `docker-compose.yaml` under `environment` section
- Read by application startup scripts

**3. Startup Arguments**
- Python services accept command-line arguments
- Flask runs with host `0.0.0.0` and specified port
- Nginx configured via configuration file in container

**4. Configuration Files**
- Nginx config embedded in `Dockerfile.frontend`
- Flask app configuration in Python source code
- Database initialization handled by Flask ORM

---

## List of Containers Used

### Container Summary Table

| Container Name | Image Type | Base Image | Port | Purpose | Service |
|---|---|---|---|---|---|
| `network-probe` | Custom | python:3.9-slim | Internal | Network monitoring probe sending ICMP pings | probe |
| `network-backend` | Custom | python:3.9-slim | 5001 | Flask API receiving probe data and managing database | backend |
| `network-frontend` | Custom | nginx:alpine | 5000 | Web server serving dashboard UI and assets | frontend |

### Detailed Container Descriptions

**1. network-probe (Probe Service)**

*Description:* The probe container continuously monitors network connectivity by sending ICMP ping requests to predefined external targets and reporting results to the backend.

*Functionality:*
- Runs Python application that implements probing logic
- Sends HTTP POST requests to backend API with monitoring data
- Executes ping operations to DNS servers (8.8.8.8, 1.1.1.1, 9.9.9.9)
- Measures response latency in milliseconds
- Handles unreachable targets gracefully
- Reports monitoring data every 10 seconds

*Image Contents:*
- Python 3.9 runtime
- `requests` library for HTTP communications
- Standard system utilities for network operations
- Minimal footprint for efficient resource usage

**2. network-backend (Backend Service)**

*Description:* The backend container provides a RESTful API for receiving monitoring data from the probe and serving it to the frontend while maintaining persistent storage.

*Functionality:*
- Runs Flask web framework application
- Exposes `/metrics` endpoint for receiving probe data (POST requests)
- Exposes `/metrics` endpoint for retrieving data (GET requests)
- Manages SQLite database (`monitoring.db`) on persistent volume
- Stores monitoring records with timestamps
- Returns latest monitoring status to frontend clients
- Handles data validation and error conditions

*Image Contents:*
- Python 3.9 runtime
- Flask web framework
- SQLite3 database library
- Additional Python dependencies as needed

**3. network-frontend (Frontend Service)**

*Description:* The frontend container serves a web-based dashboard that displays real-time network monitoring data fetched from the backend API.

*Functionality:*
- Runs Nginx web server for high performance
- Serves static HTML/CSS/JavaScript files
- Hosts dashboard UI accessible at port 5000
- Implements JavaScript fetch API calls to backend
- Displays monitoring status with color coding
- Refreshes data at regular intervals (typically every 5 seconds)
- Responsive design suitable for different screen sizes

*Image Contents:*
- Nginx Alpine Linux distribution (minimal base)
- Nginx web server configuration
- HTML/CSS dashboard files
- JavaScript code for frontend logic and API communication

---

## Instructions

### 1. Preparing the Application

**Command:** `./prepare-app.sh`

**What it does:**
- Creates custom Docker bridge network `monitoring-network`
- Creates named volume `monitoring-data` for database persistence
- Builds three custom Docker images from Dockerfiles:
  - `network-probe:latest` from `Dockerfile.probe`
  - `network-backend:latest` from `Dockerfile.backend`
  - `network-frontend:latest` from `Dockerfile.frontend`
- Performs any necessary initialization tasks

**Prerequisites:**
- Docker daemon must be running
- Sufficient disk space for image layers (~1.5 GB)
- User must have Docker permissions

**Expected Output:**
```
Preparing app...
Creating network: monitoring-network
Creating volume: monitoring-data
Building probe image...
Building backend image...
Building frontend image...
Setup complete!
```

**Time Required:** 2-5 minutes (varies based on system performance and internet speed)

---

### 2. Launching the Application

**Command:** `./start-app.sh`

**What it does:**
- Starts all three containers in detached mode
- Configures automatic restart on failure (`unless-stopped` policy)
- Establishes communication between containers via Docker network
- Mounts persistent volume to backend container
- Maps port 5000 from frontend container to host

**Prerequisites:**
- `prepare-app.sh` must have been run successfully
- Port 5000 must be available on host machine
- Docker daemon must be running

**Expected Output:**
```
Running app...
Starting probe container...
Starting backend container...
Starting frontend container...
The app is available at http://localhost:5000
```

**Startup Sequence:**
1. Backend container starts first and initializes database
2. Frontend container starts and serves dashboard
3. Probe container starts and begins monitoring

**Verification:**
- Check running containers: `docker ps`
- Access dashboard: Open browser to `http://localhost:5000`
- Should see network monitoring dashboard with target status

**Time Required:** 30-60 seconds for all containers to be ready

---

### 3. Pausing the Application

**Command:** `./stop-app.sh`

**What it does:**
- Gracefully stops all running containers
- Preserves all data in named volumes
- Preserves Docker network configuration
- Allows containers to be restarted without data loss

**Important Notes:**
- Containers are **stopped** but not **removed**
- Database data persists in named volume
- Network and volume remain available
- Paused containers consume minimal resources

**Expected Output:**
```
Stopping app...
Stopping probe container...
Stopping backend container...
Stopping frontend container...
All containers stopped.
```

**Data Preservation:**
- Application state is maintained in database
- Monitoring history is preserved
- Restarting with `start-app.sh` resumes with previous state

**Verification:**
- Check stopped containers: `docker ps -a`
- Verify volume still exists: `docker volume ls | grep monitoring-data`
- Verify network still exists: `docker network ls | grep monitoring-network`

**Time Required:** 10-20 seconds

---

### 4. Resuming After Pause

**Command:** `./start-app.sh` (same as initial launch)

**Behavior:**
- Restarts existing containers (not creating new ones)
- Reconnects containers to existing network
- Mounts existing volume with preserved data
- Previous monitoring data is immediately available

**No Data Loss:**
- All metrics collected before pause are retained
- Database state is exactly as it was before stopping
- Frontend dashboard displays previous monitoring history

---

### 5. Deleting the Application

**Command:** `./remove-app.sh`

**What it does:**
- Stops all running containers
- Removes all stopped containers
- Removes custom Docker network `monitoring-network`
- Removes named volume `monitoring-data` (database deleted)
- Removes custom Docker images

**Important:** This command is **destructive** and **cannot be undone**:
- All monitoring data is permanently deleted
- All containers are removed
- Network and volumes are removed
- Only images can be rebuilt by running `prepare-app.sh`

**Expected Output:**
```
Removing app...
Stopping containers...
Removing containers...
Removing network...
Removing volume...
Removing images...
Removed app. All traces deleted.
```

**After Removal:**
- System is returned to clean state
- To run application again, start with `prepare-app.sh`
- No artifacts or data remain

**Caution:** Before running this command, ensure:
- You have backed up any critical monitoring data
- You will not need historical metrics
- This is your intention (no confirmation prompt)

---

## Viewing the Application in Web Browser

### Access Method

1. **Ensure Application is Running**
   ```bash
   docker ps  # Should show three running containers
   ```

2. **Open Web Browser**
   - Use any modern web browser (Chrome, Firefox, Safari, Edge)
   - Recommended: Chrome 90+, Firefox 88+, Safari 14+

3. **Navigate to Dashboard**
   - URL: `http://localhost:5000`
   - Alternative: `http://127.0.0.1:5000`

### Dashboard Features

**Main Display:**
- Table showing monitored DNS servers (targets)
- Current status of each target (reachable/unreachable)
- Last measured latency (response time in milliseconds)
- Last update timestamp

**Target Information:**
- Target IP Address
- Target Name (e.g., "Google DNS")
- Current Status Badge
  - Green: Reachable (last ping successful)
  - Red: Unreachable (last ping failed)
- Latency Value
- Status Message

**Auto-Refresh:**
- Dashboard automatically updates every 5 seconds
- Data fetched from backend API
- No manual refresh needed

### Troubleshooting Access

**If Page Won't Load:**
1. Verify containers are running: `docker ps`
2. Check frontend logs: `docker logs network-frontend`
3. Verify port 5000 is listening: `netstat -tlnp | grep 5000`

**If Data Shows as Unavailable:**
1. Verify backend container is running: `docker ps | grep backend`
2. Check backend logs: `docker logs network-backend`
3. Wait 10-15 seconds for first probe results

**If Port 5000 is Already in Use:**
1. Find what's using port 5000: `lsof -i :5000` or `netstat -tlnp`
2. Either stop that service or use port mapping modification

---

### AI Assistance Used in This Project

**1. Flask Backend Development**
- AI generated RESTful API endpoints
- Database schema design and SQLite integration
- Error handling and data validation logic

**2. Frontend Dashboard**
- AI generated HTML/CSS dashboard template
- JavaScript fetch API implementation
- Real-time data refresh and status visualization

**3. Documentation**
- Technical accuracy and completeness