513 lines
16 KiB
Markdown
513 lines
16 KiB
Markdown
# Network Monitoring System - Docker Multi-Container Application
|
|
|
|
## Overview
|
|
|
|
This project implements a lightweight **Network Monitoring System** using a multi-container Docker architecture. The application continuously monitors network connectivity and latency to predefined targets (public DNS servers) and displays real-time monitoring data through an interactive web dashboard.
|
|
|
|
The system demonstrates core Docker concepts including:
|
|
- Multi-container orchestration with Docker Compose
|
|
- Custom Docker image creation from Dockerfiles
|
|
- Container networking and inter-service communication
|
|
- Persistent data storage using Docker named volumes
|
|
- Service isolation and containerization
|
|
- Port mapping and exposure control
|
|
|
|
---
|
|
|
|
## Application Description
|
|
|
|
### Purpose
|
|
|
|
The Network Monitoring System is a simplified Network Operations Center (NOC) that provides real-time visibility into network connectivity and performance metrics. It continuously probes network targets and visualizes the results through a web dashboard.
|
|
|
|
### Functionality
|
|
|
|
**Core Features:**
|
|
|
|
1. **Network Probing**
|
|
- Continuous ICMP ping requests to multiple targets
|
|
- Monitoring targets include:
|
|
- Google DNS: `8.8.8.8`
|
|
- Cloudflare DNS: `1.1.1.1`
|
|
- Quad9 DNS: `9.9.9.9`
|
|
- Measures network latency (response time in milliseconds)
|
|
- Detects host reachability and availability
|
|
|
|
2. **Data Processing & Storage**
|
|
- Backend API receives probe data
|
|
- Processes and validates monitoring metrics
|
|
- Stores data persistently in SQLite database
|
|
- Maintains historical records across container restarts
|
|
|
|
3. **Web Dashboard**
|
|
- Real-time visualization of network status
|
|
- Shows last known state of each monitored target
|
|
- Displays latency metrics and availability status
|
|
- Accessible via web browser at `http://localhost:5000`
|
|
|
|
### System Behavior
|
|
|
|
- Probe service runs network checks every 10 seconds
|
|
- Backend API listens for probe updates and processes them
|
|
- Frontend dashboard updates to reflect latest monitoring data
|
|
- Data persists even if containers are stopped and restarted
|
|
- All services communicate over isolated Docker network
|
|
|
|
---
|
|
|
|
## Virtual Networks and Named Volumes
|
|
|
|
### Docker Network Architecture
|
|
|
|
**Network Name:** `noc-net`
|
|
|
|
**Type:** Custom Bridge Network (user-defined)
|
|
|
|
**Purpose:** Isolates application containers from the host and enables service-to-service communication
|
|
|
|
**Connected Services:**
|
|
- `probe` service: Generates monitoring data
|
|
- `backend` service: Processes and stores data
|
|
- `frontend` service: Displays data via web interface
|
|
|
|
**Communication Features:**
|
|
- Automatic DNS resolution: Services reference each other by name
|
|
- Isolated from default Docker bridge network
|
|
- Host machine accesses frontend via port mapping: `5000:5000`
|
|
|
|
**Network Topology:**
|
|
|
|
```
|
|
┌─────────────────────────────────────┐
|
|
│ monitoring-network (Bridge) │
|
|
│ │
|
|
│ ┌─────────────┐ ┌─────────────┐ │
|
|
│ │ probe │ │ backend │ │
|
|
│ │ container │──│ container │ │
|
|
│ │ │ │ │ │
|
|
│ └─────────────┘ └─────────────┘ │
|
|
│ ▲ │
|
|
│ │ │
|
|
│ ┌─────────────┐ │ │
|
|
│ │ frontend │─────────┘ │
|
|
│ │ container │ │
|
|
│ │ │ │
|
|
│ └─────────────┘ │
|
|
│ │ (port 5000) │
|
|
│ └─────────────────────────── │
|
|
└─────────────────────────────────────┘
|
|
│
|
|
│ (exposed to host)
|
|
▼
|
|
localhost:5000
|
|
```
|
|
|
|
### Persistent Volumes
|
|
|
|
**Volume Name:** `noc-data`
|
|
|
|
**Type:** Named Volume (Docker-managed)
|
|
|
|
**Mount Path:** `/data` inside backend container
|
|
|
|
**Purpose:** Persists SQLite database file across container lifecycle events
|
|
|
|
**Data Preservation:**
|
|
- Survives container stop/start cycles
|
|
- Survives container removal (volume independent of container)
|
|
- Accessible by backend service for read/write operations
|
|
- Database remains intact when application is paused
|
|
|
|
**Volume Details:**
|
|
- Driver: local
|
|
- Capacity: Limited by host filesystem
|
|
- Backup: Can be backed up using `docker volume inspect` and `docker run` commands
|
|
|
|
---
|
|
|
|
## Container Configuration
|
|
|
|
### Container Specifications
|
|
|
|
#### 1. Probe Container
|
|
- **Image:** Custom image built from `Dockerfile.probe`
|
|
- **Container Name:** `network-probe`
|
|
- **Port Mapping:** Internal only (not exposed to host)
|
|
- **Network:** monitoring-network
|
|
- **Restart Policy:** `unless-stopped`
|
|
- **Environment Variables:**
|
|
- `BACKEND_URL`: Set to `http://backend:5001` for internal communication
|
|
- `PROBE_INTERVAL`: Interval between probes (5 seconds)
|
|
- **Volumes:** None
|
|
- **Resources:** No specific limits defined (uses host defaults)
|
|
- **Dependencies:** Requires `backend` service to be running
|
|
|
|
#### 2. Backend Container
|
|
- **Image:** Custom image built from `Dockerfile.backend`
|
|
- **Container Name:** `network-backend`
|
|
- **Port Mapping:** `5001:5001` (internal network only)
|
|
- **Network:** monitoring-network
|
|
- **Restart Policy:** `unless-stopped`
|
|
- **Environment Variables:**
|
|
- `FLASK_ENV`: Set to `development`
|
|
- `DATABASE_PATH`: `/data/monitoring.db`
|
|
- **Volumes:**
|
|
- Named volume `monitoring-data` mounted at `/data`
|
|
- **Resources:** No specific limits defined
|
|
- **Startup:** Flask API server on port 5001
|
|
- **Dependencies:** None (but required by probe and frontend)
|
|
|
|
#### 3. Frontend Container
|
|
- **Image:** Custom image built from `Dockerfile.frontend`
|
|
- **Container Name:** `network-frontend`
|
|
- **Port Mapping:** `5000:5000` (exposed to host)
|
|
- **Network:** monitoring-network
|
|
- **Restart Policy:** `unless-stopped`
|
|
- **Environment Variables:**
|
|
- `BACKEND_API_URL`: Set to `http://backend:5001` for internal communication
|
|
- **Volumes:** None
|
|
- **Resources:** No specific limits defined
|
|
- **Startup:** Static file server on port 5000
|
|
- **Dependencies:** Requires `backend` service for API calls
|
|
|
|
|
|
### Configuration Methods
|
|
|
|
**1. Docker Compose (Primary Method)**
|
|
- All configurations defined in `docker-compose.yaml`
|
|
- Environment variables specified in service definitions
|
|
- Volume mounts declared in service sections
|
|
- Network configuration defined at compose level
|
|
|
|
**2. Environment Variables**
|
|
- Used for dynamic configuration without rebuilding images
|
|
- Set in `docker-compose.yaml` under `environment` section
|
|
- Read by application startup scripts
|
|
|
|
**3. Startup Arguments**
|
|
- Python services accept command-line arguments
|
|
- Flask runs with host `0.0.0.0` and specified port
|
|
- Nginx configured via configuration file in container
|
|
|
|
**4. Configuration Files**
|
|
- Nginx config embedded in `Dockerfile.frontend`
|
|
- Flask app configuration in Python source code
|
|
- Database initialization handled by Flask ORM
|
|
|
|
---
|
|
|
|
## List of Containers Used
|
|
|
|
### Container Summary Table
|
|
|
|
| Container Name | Image Type | Base Image | Port | Purpose | Service |
|
|
|---|---|---|---|---|---|
|
|
| `network-probe` | Custom | python:3.9-slim | Internal | Network monitoring probe sending ICMP pings | probe |
|
|
| `network-backend` | Custom | python:3.9-slim | 5001 | Flask API receiving probe data and managing database | backend |
|
|
| `network-frontend` | Custom | nginx:alpine | 5000 | Web server serving dashboard UI and assets | frontend |
|
|
|
|
### Detailed Container Descriptions
|
|
|
|
**1. network-probe (Probe Service)**
|
|
|
|
*Description:* The probe container continuously monitors network connectivity by sending ICMP ping requests to predefined external targets and reporting results to the backend.
|
|
|
|
*Functionality:*
|
|
- Runs Python application that implements probing logic
|
|
- Sends HTTP POST requests to backend API with monitoring data
|
|
- Executes ping operations to DNS servers (8.8.8.8, 1.1.1.1, 9.9.9.9)
|
|
- Measures response latency in milliseconds
|
|
- Handles unreachable targets gracefully
|
|
- Reports monitoring data every 10 seconds
|
|
|
|
*Image Contents:*
|
|
- Python 3.9 runtime
|
|
- `requests` library for HTTP communications
|
|
- Standard system utilities for network operations
|
|
- Minimal footprint for efficient resource usage
|
|
|
|
**2. network-backend (Backend Service)**
|
|
|
|
*Description:* The backend container provides a RESTful API for receiving monitoring data from the probe and serving it to the frontend while maintaining persistent storage.
|
|
|
|
*Functionality:*
|
|
- Runs Flask web framework application
|
|
- Exposes `/metrics` endpoint for receiving probe data (POST requests)
|
|
- Exposes `/metrics` endpoint for retrieving data (GET requests)
|
|
- Manages SQLite database (`monitoring.db`) on persistent volume
|
|
- Stores monitoring records with timestamps
|
|
- Returns latest monitoring status to frontend clients
|
|
- Handles data validation and error conditions
|
|
|
|
*Image Contents:*
|
|
- Python 3.9 runtime
|
|
- Flask web framework
|
|
- SQLite3 database library
|
|
- Additional Python dependencies as needed
|
|
|
|
**3. network-frontend (Frontend Service)**
|
|
|
|
*Description:* The frontend container serves a web-based dashboard that displays real-time network monitoring data fetched from the backend API.
|
|
|
|
*Functionality:*
|
|
- Runs Nginx web server for high performance
|
|
- Serves static HTML/CSS/JavaScript files
|
|
- Hosts dashboard UI accessible at port 5000
|
|
- Implements JavaScript fetch API calls to backend
|
|
- Displays monitoring status with color coding
|
|
- Refreshes data at regular intervals (typically every 5 seconds)
|
|
- Responsive design suitable for different screen sizes
|
|
|
|
*Image Contents:*
|
|
- Nginx Alpine Linux distribution (minimal base)
|
|
- Nginx web server configuration
|
|
- HTML/CSS dashboard files
|
|
- JavaScript code for frontend logic and API communication
|
|
|
|
---
|
|
|
|
## Instructions
|
|
|
|
### 1. Preparing the Application
|
|
|
|
**Command:** `./prepare-app.sh`
|
|
|
|
**What it does:**
|
|
- Creates custom Docker bridge network `monitoring-network`
|
|
- Creates named volume `monitoring-data` for database persistence
|
|
- Builds three custom Docker images from Dockerfiles:
|
|
- `network-probe:latest` from `Dockerfile.probe`
|
|
- `network-backend:latest` from `Dockerfile.backend`
|
|
- `network-frontend:latest` from `Dockerfile.frontend`
|
|
- Performs any necessary initialization tasks
|
|
|
|
**Prerequisites:**
|
|
- Docker daemon must be running
|
|
- Sufficient disk space for image layers (~1.5 GB)
|
|
- User must have Docker permissions
|
|
|
|
**Expected Output:**
|
|
```
|
|
Preparing app...
|
|
Creating network: monitoring-network
|
|
Creating volume: monitoring-data
|
|
Building probe image...
|
|
Building backend image...
|
|
Building frontend image...
|
|
Setup complete!
|
|
```
|
|
|
|
**Time Required:** 2-5 minutes (varies based on system performance and internet speed)
|
|
|
|
---
|
|
|
|
### 2. Launching the Application
|
|
|
|
**Command:** `./start-app.sh`
|
|
|
|
**What it does:**
|
|
- Starts all three containers in detached mode
|
|
- Configures automatic restart on failure (`unless-stopped` policy)
|
|
- Establishes communication between containers via Docker network
|
|
- Mounts persistent volume to backend container
|
|
- Maps port 5000 from frontend container to host
|
|
|
|
**Prerequisites:**
|
|
- `prepare-app.sh` must have been run successfully
|
|
- Port 5000 must be available on host machine
|
|
- Docker daemon must be running
|
|
|
|
**Expected Output:**
|
|
```
|
|
Running app...
|
|
Starting probe container...
|
|
Starting backend container...
|
|
Starting frontend container...
|
|
The app is available at http://localhost:5000
|
|
```
|
|
|
|
**Startup Sequence:**
|
|
1. Backend container starts first and initializes database
|
|
2. Frontend container starts and serves dashboard
|
|
3. Probe container starts and begins monitoring
|
|
|
|
**Verification:**
|
|
- Check running containers: `docker ps`
|
|
- Access dashboard: Open browser to `http://localhost:5000`
|
|
- Should see network monitoring dashboard with target status
|
|
|
|
**Time Required:** 30-60 seconds for all containers to be ready
|
|
|
|
---
|
|
|
|
### 3. Pausing the Application
|
|
|
|
**Command:** `./stop-app.sh`
|
|
|
|
**What it does:**
|
|
- Gracefully stops all running containers
|
|
- Preserves all data in named volumes
|
|
- Preserves Docker network configuration
|
|
- Allows containers to be restarted without data loss
|
|
|
|
**Important Notes:**
|
|
- Containers are **stopped** but not **removed**
|
|
- Database data persists in named volume
|
|
- Network and volume remain available
|
|
- Paused containers consume minimal resources
|
|
|
|
**Expected Output:**
|
|
```
|
|
Stopping app...
|
|
Stopping probe container...
|
|
Stopping backend container...
|
|
Stopping frontend container...
|
|
All containers stopped.
|
|
```
|
|
|
|
**Data Preservation:**
|
|
- Application state is maintained in database
|
|
- Monitoring history is preserved
|
|
- Restarting with `start-app.sh` resumes with previous state
|
|
|
|
**Verification:**
|
|
- Check stopped containers: `docker ps -a`
|
|
- Verify volume still exists: `docker volume ls | grep monitoring-data`
|
|
- Verify network still exists: `docker network ls | grep monitoring-network`
|
|
|
|
**Time Required:** 10-20 seconds
|
|
|
|
---
|
|
|
|
### 4. Resuming After Pause
|
|
|
|
**Command:** `./start-app.sh` (same as initial launch)
|
|
|
|
**Behavior:**
|
|
- Restarts existing containers (not creating new ones)
|
|
- Reconnects containers to existing network
|
|
- Mounts existing volume with preserved data
|
|
- Previous monitoring data is immediately available
|
|
|
|
**No Data Loss:**
|
|
- All metrics collected before pause are retained
|
|
- Database state is exactly as it was before stopping
|
|
- Frontend dashboard displays previous monitoring history
|
|
|
|
---
|
|
|
|
### 5. Deleting the Application
|
|
|
|
**Command:** `./remove-app.sh`
|
|
|
|
**What it does:**
|
|
- Stops all running containers
|
|
- Removes all stopped containers
|
|
- Removes custom Docker network `monitoring-network`
|
|
- Removes named volume `monitoring-data` (database deleted)
|
|
- Removes custom Docker images
|
|
|
|
**Important:** This command is **destructive** and **cannot be undone**:
|
|
- All monitoring data is permanently deleted
|
|
- All containers are removed
|
|
- Network and volumes are removed
|
|
- Only images can be rebuilt by running `prepare-app.sh`
|
|
|
|
**Expected Output:**
|
|
```
|
|
Removing app...
|
|
Stopping containers...
|
|
Removing containers...
|
|
Removing network...
|
|
Removing volume...
|
|
Removing images...
|
|
Removed app. All traces deleted.
|
|
```
|
|
|
|
**After Removal:**
|
|
- System is returned to clean state
|
|
- To run application again, start with `prepare-app.sh`
|
|
- No artifacts or data remain
|
|
|
|
**Caution:** Before running this command, ensure:
|
|
- You have backed up any critical monitoring data
|
|
- You will not need historical metrics
|
|
- This is your intention (no confirmation prompt)
|
|
|
|
---
|
|
|
|
## Viewing the Application in Web Browser
|
|
|
|
### Access Method
|
|
|
|
1. **Ensure Application is Running**
|
|
```bash
|
|
docker ps # Should show three running containers
|
|
```
|
|
|
|
2. **Open Web Browser**
|
|
- Use any modern web browser (Chrome, Firefox, Safari, Edge)
|
|
- Recommended: Chrome 90+, Firefox 88+, Safari 14+
|
|
|
|
3. **Navigate to Dashboard**
|
|
- URL: `http://localhost:5000`
|
|
- Alternative: `http://127.0.0.1:5000`
|
|
|
|
### Dashboard Features
|
|
|
|
**Main Display:**
|
|
- Table showing monitored DNS servers (targets)
|
|
- Current status of each target (reachable/unreachable)
|
|
- Last measured latency (response time in milliseconds)
|
|
- Last update timestamp
|
|
|
|
**Target Information:**
|
|
- Target IP Address
|
|
- Target Name (e.g., "Google DNS")
|
|
- Current Status Badge
|
|
- Green: Reachable (last ping successful)
|
|
- Red: Unreachable (last ping failed)
|
|
- Latency Value
|
|
- Status Message
|
|
|
|
**Auto-Refresh:**
|
|
- Dashboard automatically updates every 5 seconds
|
|
- Data fetched from backend API
|
|
- No manual refresh needed
|
|
|
|
### Troubleshooting Access
|
|
|
|
**If Page Won't Load:**
|
|
1. Verify containers are running: `docker ps`
|
|
2. Check frontend logs: `docker logs network-frontend`
|
|
3. Verify port 5000 is listening: `netstat -tlnp | grep 5000`
|
|
|
|
**If Data Shows as Unavailable:**
|
|
1. Verify backend container is running: `docker ps | grep backend`
|
|
2. Check backend logs: `docker logs network-backend`
|
|
3. Wait 10-15 seconds for first probe results
|
|
|
|
**If Port 5000 is Already in Use:**
|
|
1. Find what's using port 5000: `lsof -i :5000` or `netstat -tlnp`
|
|
2. Either stop that service or use port mapping modification
|
|
|
|
---
|
|
|
|
### AI Assistance Used in This Project
|
|
|
|
**1. Flask Backend Development**
|
|
- AI generated RESTful API endpoints
|
|
- Database schema design and SQLite integration
|
|
- Error handling and data validation logic
|
|
|
|
**2. Frontend Dashboard**
|
|
- AI generated HTML/CSS dashboard template
|
|
- JavaScript fetch API implementation
|
|
- Real-time data refresh and status visualization
|
|
|
|
**3. Documentation**
|
|
- Technical accuracy and completeness
|
|
|
|
|
|
|