06 Apr 2025

HTTOP - A real-time HTTP log monitor similar to ApacheTop

Category projects

Tags #rust #terminal #console #tools

A real-time HTTP log monitor similar to ApacheTop, written in Rust. This tool allows you to watch incoming http requests in real-time. Purpose - research on web crawlers.

Features

Processes http log data in real-time through pipe from tail -f
Displays overall statistics (requests per second, total bytes, status code distribution)

Roadmap

Filtering by Crawler bots to research on crawlers
Indicate “bad” crawlers

Installation

Prerequisites

Rust and Cargo (install from rust-lang.org)
Nginx server with access logs

Building from Source

Clone the repository:

git clone https://github.com/yourusername/nginx-top.git
cd nginx-top

Build the project:
```
cargo build --release
```
The compiled binary will be available at target/release/nginx-top

Usage

Pipe Nginx log data to NginxTop:

tail -f /var/log/nginx/access.log | httop

Interactive Controls

Type the following characters and press Enter to control the display:

s: Sort by Status Code
p: Sort by Path
c: Sort by Count (default)
i: Sort by IP Address
u: Sort by User Agent
+: Increase number of displayed entries
-: Decrease number of displayed entries
q: Quit

Nginx Log Format Compatibility

NginxTop is configured to parse the standard Nginx log format:

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                '$status $body_bytes_sent "$http_referer" '
                '"$http_user_agent" $request_time';

If your Nginx uses a different log format, you may need to modify the regex pattern in the parse_log_line function.

Sample Output

Total Requests: 1548 | RPS: 32.50 | Total Bytes: 28945213

Status Codes:
  200: 1423
  404: 87
  302: 34
  500: 4

Top Requests (Sort: Count, Press s/p/c/i/u to change, +/- to adjust count, q to quit):

COUNT    IP              STATUS     PATH                                    USER AGENT
-------  --------------  ---------  --------------------------------------  -----------------------------------
183      192.168.1.45    200        /index.html                             Mozilla/5.0 (Windows NT 10.0; Win...
127      192.168.1.22    200        /assets/css/main.css                    Mozilla/5.0 (Macintosh; Intel Mac...
98       192.168.1.60    200        /assets/js/app.js                       Mozilla/5.0 (iPhone; CPU iPhone O...
76       192.168.1.33    200        /api/users                              PostmanRuntime/7.29.0

License

GNU v3

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Fork the project
Create your feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add some amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request