sb693yd/ZKT26
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
main
ZKT26/noc-docker/README.md

16 KiB
Raw Permalink Blame History

Network Monitoring System - Docker Multi-Container Application

Overview

This project implements a lightweight Network Monitoring System using a multi-container Docker architecture. The application continuously monitors network connectivity and latency to predefined targets (public DNS servers) and displays real-time monitoring data through an interactive web dashboard.

The system demonstrates core Docker concepts including:

  • Multi-container orchestration with Docker Compose
  • Custom Docker image creation from Dockerfiles
  • Container networking and inter-service communication
  • Persistent data storage using Docker named volumes
  • Service isolation and containerization
  • Port mapping and exposure control

Application Description

Purpose

The Network Monitoring System is a simplified Network Operations Center (NOC) that provides real-time visibility into network connectivity and performance metrics. It continuously probes network targets and visualizes the results through a web dashboard.

Functionality

Core Features:

  1. Network Probing

    • Continuous ICMP ping requests to multiple targets
    • Monitoring targets include:
      • Google DNS: 8.8.8.8
      • Cloudflare DNS: 1.1.1.1
      • Quad9 DNS: 9.9.9.9
    • Measures network latency (response time in milliseconds)
    • Detects host reachability and availability

  2. Data Processing & Storage

    • Backend API receives probe data
    • Processes and validates monitoring metrics
    • Stores data persistently in SQLite database
    • Maintains historical records across container restarts

  3. Web Dashboard

    • Real-time visualization of network status
    • Shows last known state of each monitored target
    • Displays latency metrics and availability status
    • Accessible via web browser at http://localhost:5000

System Behavior

  • Probe service runs network checks every 10 seconds
  • Backend API listens for probe updates and processes them
  • Frontend dashboard updates to reflect latest monitoring data
  • Data persists even if containers are stopped and restarted
  • All services communicate over isolated Docker network

Virtual Networks and Named Volumes

Docker Network Architecture

Network Name: noc-net

Type: Custom Bridge Network (user-defined)

Purpose: Isolates application containers from the host and enables service-to-service communication

Connected Services:

  • probe service: Generates monitoring data
  • backend service: Processes and stores data
  • frontend service: Displays data via web interface

Communication Features:

  • Automatic DNS resolution: Services reference each other by name
  • Isolated from default Docker bridge network
  • Host machine accesses frontend via port mapping: 5000:5000

Network Topology:

┌─────────────────────────────────────┐
│    monitoring-network (Bridge)      │
│                                     │
│  ┌─────────────┐  ┌─────────────┐  │
│  │   probe     │  │  backend    │  │
│  │  container  │──│  container  │  │
│  │             │  │             │  │
│  └─────────────┘  └─────────────┘  │
│                          ▲           │
│                          │           │
│  ┌─────────────┐         │           │
│  │  frontend   │─────────┘           │
│  │  container  │                     │
│  │             │                     │
│  └─────────────┘                     │
│         │ (port 5000)                │
│         └─────────────────────────── │
└─────────────────────────────────────┘
              │
              │ (exposed to host)
              ▼
         localhost:5000

Persistent Volumes

Volume Name: noc-data

Type: Named Volume (Docker-managed)

Mount Path: /data inside backend container

Purpose: Persists SQLite database file across container lifecycle events

Data Preservation:

  • Survives container stop/start cycles
  • Survives container removal (volume independent of container)
  • Accessible by backend service for read/write operations
  • Database remains intact when application is paused

Volume Details:

  • Driver: local
  • Capacity: Limited by host filesystem
  • Backup: Can be backed up using docker volume inspect and docker run commands

Container Configuration

Container Specifications

1. Probe Container

  • Image: Custom image built from Dockerfile.probe
  • Container Name: network-probe
  • Port Mapping: Internal only (not exposed to host)
  • Network: monitoring-network
  • Restart Policy: unless-stopped
  • Environment Variables:
    • BACKEND_URL: Set to http://backend:5001 for internal communication
    • PROBE_INTERVAL: Interval between probes (5 seconds)
  • Volumes: None
  • Resources: No specific limits defined (uses host defaults)
  • Dependencies: Requires backend service to be running

2. Backend Container

  • Image: Custom image built from Dockerfile.backend
  • Container Name: network-backend
  • Port Mapping: 5001:5001 (internal network only)
  • Network: monitoring-network
  • Restart Policy: unless-stopped
  • Environment Variables:
    • FLASK_ENV: Set to development
    • DATABASE_PATH: /data/monitoring.db
  • Volumes:
    • Named volume monitoring-data mounted at /data
  • Resources: No specific limits defined
  • Startup: Flask API server on port 5001
  • Dependencies: None (but required by probe and frontend)

3. Frontend Container

  • Image: Custom image built from Dockerfile.frontend
  • Container Name: network-frontend
  • Port Mapping: 5000:5000 (exposed to host)
  • Network: monitoring-network
  • Restart Policy: unless-stopped
  • Environment Variables:
    • BACKEND_API_URL: Set to http://backend:5001 for internal communication
  • Volumes: None
  • Resources: No specific limits defined
  • Startup: Static file server on port 5000
  • Dependencies: Requires backend service for API calls

Configuration Methods

1. Docker Compose (Primary Method)

  • All configurations defined in docker-compose.yaml
  • Environment variables specified in service definitions
  • Volume mounts declared in service sections
  • Network configuration defined at compose level

2. Environment Variables

  • Used for dynamic configuration without rebuilding images
  • Set in docker-compose.yaml under environment section
  • Read by application startup scripts

3. Startup Arguments

  • Python services accept command-line arguments
  • Flask runs with host 0.0.0.0 and specified port
  • Nginx configured via configuration file in container

4. Configuration Files

  • Nginx config embedded in Dockerfile.frontend
  • Flask app configuration in Python source code
  • Database initialization handled by Flask ORM

List of Containers Used

Container Summary Table

Container Name Image Type Base Image Port Purpose Service
network-probe Custom python:3.9-slim Internal Network monitoring probe sending ICMP pings probe
network-backend Custom python:3.9-slim 5001 Flask API receiving probe data and managing database backend
network-frontend Custom nginx:alpine 5000 Web server serving dashboard UI and assets frontend

Detailed Container Descriptions

1. network-probe (Probe Service)

Description: The probe container continuously monitors network connectivity by sending ICMP ping requests to predefined external targets and reporting results to the backend.

Functionality:

  • Runs Python application that implements probing logic
  • Sends HTTP POST requests to backend API with monitoring data
  • Executes ping operations to DNS servers (8.8.8.8, 1.1.1.1, 9.9.9.9)
  • Measures response latency in milliseconds
  • Handles unreachable targets gracefully
  • Reports monitoring data every 10 seconds

Image Contents:

  • Python 3.9 runtime
  • requests library for HTTP communications
  • Standard system utilities for network operations
  • Minimal footprint for efficient resource usage

2. network-backend (Backend Service)

Description: The backend container provides a RESTful API for receiving monitoring data from the probe and serving it to the frontend while maintaining persistent storage.

Functionality:

  • Runs Flask web framework application
  • Exposes /metrics endpoint for receiving probe data (POST requests)
  • Exposes /metrics endpoint for retrieving data (GET requests)
  • Manages SQLite database (monitoring.db) on persistent volume
  • Stores monitoring records with timestamps
  • Returns latest monitoring status to frontend clients
  • Handles data validation and error conditions

Image Contents:

  • Python 3.9 runtime
  • Flask web framework
  • SQLite3 database library
  • Additional Python dependencies as needed

3. network-frontend (Frontend Service)

Description: The frontend container serves a web-based dashboard that displays real-time network monitoring data fetched from the backend API.

Functionality:

  • Runs Nginx web server for high performance
  • Serves static HTML/CSS/JavaScript files
  • Hosts dashboard UI accessible at port 5000
  • Implements JavaScript fetch API calls to backend
  • Displays monitoring status with color coding
  • Refreshes data at regular intervals (typically every 5 seconds)
  • Responsive design suitable for different screen sizes

Image Contents:

  • Nginx Alpine Linux distribution (minimal base)
  • Nginx web server configuration
  • HTML/CSS dashboard files
  • JavaScript code for frontend logic and API communication

Instructions

1. Preparing the Application

Command: ./prepare-app.sh

What it does:

  • Creates custom Docker bridge network monitoring-network
  • Creates named volume monitoring-data for database persistence
  • Builds three custom Docker images from Dockerfiles:
    • network-probe:latest from Dockerfile.probe
    • network-backend:latest from Dockerfile.backend
    • network-frontend:latest from Dockerfile.frontend
  • Performs any necessary initialization tasks

Prerequisites:

  • Docker daemon must be running
  • Sufficient disk space for image layers (~1.5 GB)
  • User must have Docker permissions

Expected Output:

Preparing app...
Creating network: monitoring-network
Creating volume: monitoring-data
Building probe image...
Building backend image...
Building frontend image...
Setup complete!

Time Required: 2-5 minutes (varies based on system performance and internet speed)

2. Launching the Application

Command: ./start-app.sh

What it does:

  • Starts all three containers in detached mode
  • Configures automatic restart on failure (unless-stopped policy)
  • Establishes communication between containers via Docker network
  • Mounts persistent volume to backend container
  • Maps port 5000 from frontend container to host

Prerequisites:

  • prepare-app.sh must have been run successfully
  • Port 5000 must be available on host machine
  • Docker daemon must be running

Expected Output:

Running app...
Starting probe container...
Starting backend container...
Starting frontend container...
The app is available at http://localhost:5000

Startup Sequence:

  1. Backend container starts first and initializes database
  2. Frontend container starts and serves dashboard
  3. Probe container starts and begins monitoring

Verification:

  • Check running containers: docker ps
  • Access dashboard: Open browser to http://localhost:5000
  • Should see network monitoring dashboard with target status

Time Required: 30-60 seconds for all containers to be ready

3. Pausing the Application

Command: ./stop-app.sh

What it does:

  • Gracefully stops all running containers
  • Preserves all data in named volumes
  • Preserves Docker network configuration
  • Allows containers to be restarted without data loss

Important Notes:

  • Containers are stopped but not removed
  • Database data persists in named volume
  • Network and volume remain available
  • Paused containers consume minimal resources

Expected Output:

Stopping app...
Stopping probe container...
Stopping backend container...
Stopping frontend container...
All containers stopped.

Data Preservation:

  • Application state is maintained in database
  • Monitoring history is preserved
  • Restarting with start-app.sh resumes with previous state

Verification:

  • Check stopped containers: docker ps -a
  • Verify volume still exists: docker volume ls | grep monitoring-data
  • Verify network still exists: docker network ls | grep monitoring-network

Time Required: 10-20 seconds

4. Resuming After Pause

Command: ./start-app.sh (same as initial launch)

Behavior:

  • Restarts existing containers (not creating new ones)
  • Reconnects containers to existing network
  • Mounts existing volume with preserved data
  • Previous monitoring data is immediately available

No Data Loss:

  • All metrics collected before pause are retained
  • Database state is exactly as it was before stopping
  • Frontend dashboard displays previous monitoring history

5. Deleting the Application

Command: ./remove-app.sh

What it does:

  • Stops all running containers
  • Removes all stopped containers
  • Removes custom Docker network monitoring-network
  • Removes named volume monitoring-data (database deleted)
  • Removes custom Docker images

Important: This command is destructive and cannot be undone:

  • All monitoring data is permanently deleted
  • All containers are removed
  • Network and volumes are removed
  • Only images can be rebuilt by running prepare-app.sh

Expected Output:

Removing app...
Stopping containers...
Removing containers...
Removing network...
Removing volume...
Removing images...
Removed app. All traces deleted.

After Removal:

  • System is returned to clean state
  • To run application again, start with prepare-app.sh
  • No artifacts or data remain

Caution: Before running this command, ensure:

  • You have backed up any critical monitoring data
  • You will not need historical metrics
  • This is your intention (no confirmation prompt)

Viewing the Application in Web Browser

Access Method

  1. Ensure Application is Running

    docker ps  # Should show three running containers

  2. Open Web Browser

    • Use any modern web browser (Chrome, Firefox, Safari, Edge)
    • Recommended: Chrome 90+, Firefox 88+, Safari 14+

  3. Navigate to Dashboard

    • URL: http://localhost:5000
    • Alternative: http://127.0.0.1:5000

Dashboard Features

Main Display:

  • Table showing monitored DNS servers (targets)
  • Current status of each target (reachable/unreachable)
  • Last measured latency (response time in milliseconds)
  • Last update timestamp

Target Information:

  • Target IP Address
  • Target Name (e.g., "Google DNS")
  • Current Status Badge
    • Green: Reachable (last ping successful)
    • Red: Unreachable (last ping failed)
  • Latency Value
  • Status Message

Auto-Refresh:

  • Dashboard automatically updates every 5 seconds
  • Data fetched from backend API
  • No manual refresh needed

Troubleshooting Access

If Page Won't Load:

  1. Verify containers are running: docker ps
  2. Check frontend logs: docker logs network-frontend
  3. Verify port 5000 is listening: netstat -tlnp | grep 5000

If Data Shows as Unavailable:

  1. Verify backend container is running: docker ps | grep backend
  2. Check backend logs: docker logs network-backend
  3. Wait 10-15 seconds for first probe results

If Port 5000 is Already in Use:

  1. Find what's using port 5000: lsof -i :5000 or netstat -tlnp
  2. Either stop that service or use port mapping modification

AI Assistance Used in This Project

1. Flask Backend Development

  • AI generated RESTful API endpoints
  • Database schema design and SQLite integration
  • Error handling and data validation logic

2. Frontend Dashboard

  • AI generated HTML/CSS dashboard template
  • JavaScript fetch API implementation
  • Real-time data refresh and status visualization

3. Documentation

  • Technical accuracy and completeness