ZKT26/noc-docker

Network Monitoring System - Docker Multi-Container Application

Overview

This project implements a lightweight Network Monitoring System using a multi-container Docker architecture. The application continuously monitors network connectivity and latency to predefined targets (public DNS servers) and displays real-time monitoring data through an interactive web dashboard.

The system demonstrates core Docker concepts including:

• Multi-container orchestration with Docker Compose
• Custom Docker image creation from Dockerfiles
• Container networking and inter-service communication
• Persistent data storage using Docker named volumes
• Service isolation and containerization
• Port mapping and exposure control

---

Application Description

Purpose

The Network Monitoring System is a simplified Network Operations Center (NOC) that provides real-time visibility into network connectivity and performance metrics. It continuously probes network targets and visualizes the results through a web dashboard.

Functionality

Core Features:

1. Network Probing

   • Continuous ICMP ping requests to multiple targets
   • Monitoring targets include:
     • Google DNS: 8.8.8.8
     • Cloudflare DNS: 1.1.1.1
     • Quad9 DNS: 9.9.9.9
   • Measures network latency (response time in milliseconds)
   • Detects host reachability and availability

2. Data Processing & Storage

   • Backend API receives probe data
   • Processes and validates monitoring metrics
   • Stores data persistently in SQLite database
   • Maintains historical records across container restarts

3. Web Dashboard

   • Real-time visualization of network status
   • Shows last known state of each monitored target
   • Displays latency metrics and availability status
   • Accessible via web browser at http://localhost:5000

System Behavior

• Probe service runs network checks every 10 seconds
• Backend API listens for probe updates and processes them
• Frontend dashboard updates to reflect latest monitoring data
• Data persists even if containers are stopped and restarted
• All services communicate over isolated Docker network

---

Virtual Networks and Named Volumes

Docker Network Architecture

Network Name: noc-net

Type: Custom Bridge Network (user-defined)

Purpose: Isolates application containers from the host and enables service-to-service communication

Connected Services:

• probe service: Generates monitoring data
• backend service: Processes and stores data
• frontend service: Displays data via web interface

Communication Features:

• Automatic DNS resolution: Services reference each other by name
• Isolated from default Docker bridge network
• Host machine accesses frontend via port mapping: 5000:5000

Network Topology:

┌─────────────────────────────────────┐
│    monitoring-network (Bridge)      │
│                                     │
│  ┌─────────────┐  ┌─────────────┐  │
│  │   probe     │  │  backend    │  │
│  │  container  │──│  container  │  │
│  │             │  │             │  │
│  └─────────────┘  └─────────────┘  │
│                          ▲           │
│                          │           │
│  ┌─────────────┐         │           │
│  │  frontend   │─────────┘           │
│  │  container  │                     │
│  │             │                     │
│  └─────────────┘                     │
│         │ (port 5000)                │
│         └─────────────────────────── │
└─────────────────────────────────────┘
              │
              │ (exposed to host)
              ▼
         localhost:5000

Persistent Volumes

Volume Name: noc-data

Type: Named Volume (Docker-managed)

Mount Path: /data inside backend container

Purpose: Persists SQLite database file across container lifecycle events

Data Preservation:

• Survives container stop/start cycles
• Survives container removal (volume independent of container)
• Accessible by backend service for read/write operations
• Database remains intact when application is paused

Volume Details:

• Driver: local
• Capacity: Limited by host filesystem
• Backup: Can be backed up using docker volume inspect and docker run commands

---

Container Configuration

Container Specifications

1. Probe Container

• Image: Custom image built from Dockerfile.probe
• Container Name: network-probe
• Port Mapping: Internal only (not exposed to host)
• Network: monitoring-network
• Restart Policy: unless-stopped
• Environment Variables:
  • BACKEND_URL: Set to http://backend:5001 for internal communication
  • PROBE_INTERVAL: Interval between probes (5 seconds)
• Volumes: None
• Resources: No specific limits defined (uses host defaults)
• Dependencies: Requires backend service to be running

2. Backend Container

• Image: Custom image built from Dockerfile.backend
• Container Name: network-backend
• Port Mapping: 5001:5001 (internal network only)
• Network: monitoring-network
• Restart Policy: unless-stopped
• Environment Variables:
  • FLASK_ENV: Set to development
  • DATABASE_PATH: /data/monitoring.db
• Volumes:
  • Named volume monitoring-data mounted at /data
• Resources: No specific limits defined
• Startup: Flask API server on port 5001
• Dependencies: None (but required by probe and frontend)

3. Frontend Container

• Image: Custom image built from Dockerfile.frontend
• Container Name: network-frontend
• Port Mapping: 5000:5000 (exposed to host)
• Network: monitoring-network
• Restart Policy: unless-stopped
• Environment Variables:
  • BACKEND_API_URL: Set to http://backend:5001 for internal communication
• Volumes: None
• Resources: No specific limits defined
• Startup: Static file server on port 5000
• Dependencies: Requires backend service for API calls

Configuration Methods

1. Docker Compose (Primary Method)

• All configurations defined in docker-compose.yaml
• Environment variables specified in service definitions
• Volume mounts declared in service sections
• Network configuration defined at compose level

2. Environment Variables

• Used for dynamic configuration without rebuilding images
• Set in docker-compose.yaml under environment section
• Read by application startup scripts

3. Startup Arguments

• Python services accept command-line arguments
• Flask runs with host 0.0.0.0 and specified port
• Nginx configured via configuration file in container

4. Configuration Files

• Nginx config embedded in Dockerfile.frontend
• Flask app configuration in Python source code
• Database initialization handled by Flask ORM

---

List of Containers Used

Container Summary Table

Container Name	Image Type	Base Image	Port	Purpose	Service
network-probe	Custom	python:3.9-slim	Internal	Network monitoring probe sending ICMP pings	probe
network-backend	Custom	python:3.9-slim	5001	Flask API receiving probe data and managing database	backend
network-frontend	Custom	nginx:alpine	5000	Web server serving dashboard UI and assets	frontend

Detailed Container Descriptions

1. network-probe (Probe Service)

Description: The probe container continuously monitors network connectivity by sending ICMP ping requests to predefined external targets and reporting results to the backend.

Functionality:

• Runs Python application that implements probing logic
• Sends HTTP POST requests to backend API with monitoring data
• Executes ping operations to DNS servers (8.8.8.8, 1.1.1.1, 9.9.9.9)
• Measures response latency in milliseconds
• Handles unreachable targets gracefully
• Reports monitoring data every 10 seconds

Image Contents:

• Python 3.9 runtime
• requests library for HTTP communications
• Standard system utilities for network operations
• Minimal footprint for efficient resource usage

2. network-backend (Backend Service)

Description: The backend container provides a RESTful API for receiving monitoring data from the probe and serving it to the frontend while maintaining persistent storage.

Functionality:

• Runs Flask web framework application
• Exposes /metrics endpoint for receiving probe data (POST requests)
• Exposes /metrics endpoint for retrieving data (GET requests)
• Manages SQLite database (monitoring.db) on persistent volume
• Stores monitoring records with timestamps
• Returns latest monitoring status to frontend clients
• Handles data validation and error conditions

Image Contents:

• Python 3.9 runtime
• Flask web framework
• SQLite3 database library
• Additional Python dependencies as needed

3. network-frontend (Frontend Service)

Description: The frontend container serves a web-based dashboard that displays real-time network monitoring data fetched from the backend API.

Functionality:

• Runs Nginx web server for high performance
• Serves static HTML/CSS/JavaScript files
• Hosts dashboard UI accessible at port 5000
• Implements JavaScript fetch API calls to backend
• Displays monitoring status with color coding
• Refreshes data at regular intervals (typically every 5 seconds)
• Responsive design suitable for different screen sizes

Image Contents:

• Nginx Alpine Linux distribution (minimal base)
• Nginx web server configuration
• HTML/CSS dashboard files
• JavaScript code for frontend logic and API communication

---

Instructions

1. Preparing the Application

Command: ./prepare-app.sh

What it does:

• Creates custom Docker bridge network monitoring-network
• Creates named volume monitoring-data for database persistence
• Builds three custom Docker images from Dockerfiles:
  • network-probe:latest from Dockerfile.probe
  • network-backend:latest from Dockerfile.backend
  • network-frontend:latest from Dockerfile.frontend
• Performs any necessary initialization tasks

Prerequisites:

• Docker daemon must be running
• Sufficient disk space for image layers (~1.5 GB)
• User must have Docker permissions

Expected Output:

Preparing app...
Creating network: monitoring-network
Creating volume: monitoring-data
Building probe image...
Building backend image...
Building frontend image...
Setup complete!

Time Required: 2-5 minutes (varies based on system performance and internet speed)

---

2. Launching the Application

Command: ./start-app.sh

What it does:

• Starts all three containers in detached mode
• Configures automatic restart on failure (unless-stopped policy)
• Establishes communication between containers via Docker network
• Mounts persistent volume to backend container
• Maps port 5000 from frontend container to host

Prerequisites:

• prepare-app.sh must have been run successfully
• Port 5000 must be available on host machine
• Docker daemon must be running

Expected Output:

Running app...
Starting probe container...
Starting backend container...
Starting frontend container...
The app is available at http://localhost:5000

Startup Sequence:

1. Backend container starts first and initializes database
2. Frontend container starts and serves dashboard
3. Probe container starts and begins monitoring

Verification:

• Check running containers: docker ps
• Access dashboard: Open browser to http://localhost:5000
• Should see network monitoring dashboard with target status

Time Required: 30-60 seconds for all containers to be ready

---

3. Pausing the Application

Command: ./stop-app.sh

What it does:

• Gracefully stops all running containers
• Preserves all data in named volumes
• Preserves Docker network configuration
• Allows containers to be restarted without data loss

Important Notes:

• Containers are stopped but not removed
• Database data persists in named volume
• Network and volume remain available
• Paused containers consume minimal resources

Expected Output:

Stopping app...
Stopping probe container...
Stopping backend container...
Stopping frontend container...
All containers stopped.

Data Preservation:

• Application state is maintained in database
• Monitoring history is preserved
• Restarting with start-app.sh resumes with previous state

Verification:

• Check stopped containers: docker ps -a
• Verify volume still exists: docker volume ls | grep monitoring-data
• Verify network still exists: docker network ls | grep monitoring-network

Time Required: 10-20 seconds

---

4. Resuming After Pause

Command: ./start-app.sh (same as initial launch)

Behavior:

• Restarts existing containers (not creating new ones)
• Reconnects containers to existing network
• Mounts existing volume with preserved data
• Previous monitoring data is immediately available

No Data Loss:

• All metrics collected before pause are retained
• Database state is exactly as it was before stopping
• Frontend dashboard displays previous monitoring history

---

5. Deleting the Application

Command: ./remove-app.sh

What it does:

• Stops all running containers
• Removes all stopped containers
• Removes custom Docker network monitoring-network
• Removes named volume monitoring-data (database deleted)
• Removes custom Docker images

Important: This command is destructive and cannot be undone:

• All monitoring data is permanently deleted
• All containers are removed
• Network and volumes are removed
• Only images can be rebuilt by running prepare-app.sh

Expected Output:

Removing app...
Stopping containers...
Removing containers...
Removing network...
Removing volume...
Removing images...
Removed app. All traces deleted.

After Removal:

• System is returned to clean state
• To run application again, start with prepare-app.sh
• No artifacts or data remain

Caution: Before running this command, ensure:

• You have backed up any critical monitoring data
• You will not need historical metrics
• This is your intention (no confirmation prompt)

---

Viewing the Application in Web Browser

Access Method

1. Ensure Application is Running

   docker ps  # Should show three running containers
   

2. Open Web Browser

   • Use any modern web browser (Chrome, Firefox, Safari, Edge)
   • Recommended: Chrome 90+, Firefox 88+, Safari 14+

3. Navigate to Dashboard

   • URL: http://localhost:5000
   • Alternative: http://127.0.0.1:5000

Dashboard Features

Main Display:

• Table showing monitored DNS servers (targets)
• Current status of each target (reachable/unreachable)
• Last measured latency (response time in milliseconds)
• Last update timestamp

Target Information:

• Target IP Address
• Target Name (e.g., "Google DNS")
• Current Status Badge
  • Green: Reachable (last ping successful)
  • Red: Unreachable (last ping failed)
• Latency Value
• Status Message

Auto-Refresh:

• Dashboard automatically updates every 5 seconds
• Data fetched from backend API
• No manual refresh needed

Troubleshooting Access

If Page Won't Load:

1. Verify containers are running: docker ps
2. Check frontend logs: docker logs network-frontend
3. Verify port 5000 is listening: netstat -tlnp | grep 5000

If Data Shows as Unavailable:

1. Verify backend container is running: docker ps | grep backend
2. Check backend logs: docker logs network-backend
3. Wait 10-15 seconds for first probe results

If Port 5000 is Already in Use:

1. Find what's using port 5000: lsof -i :5000 or netstat -tlnp
2. Either stop that service or use port mapping modification

---

AI Assistance Used in This Project

1. Flask Backend Development

• AI generated RESTful API endpoints
• Database schema design and SQLite integration
• Error handling and data validation logic

2. Frontend Dashboard

• AI generated HTML/CSS dashboard template
• JavaScript fetch API implementation
• Real-time data refresh and status visualization

3. Documentation

• Technical accuracy and completeness