Log Analyzer

Script

scripts/log-analyzer.sh — the single entry point for all analyses.

The script is self-contained, works on any Linux system with standard tools (grep, awk, sort, uniq), and handles both file and piped input.

Quick Start

Analyze /var/log/syslog with all checks:

bash scripts/log-analyzer.sh -f /var/log/syslog --all

Or pipe logs directly:

journalctl -u nginx --since "24 hours ago" | bash scripts/log-analyzer.sh --all

Options

Analysis Modules

Error Patterns (--errors)

Scans for configured error patterns, groups and sorts by frequency, shows the top most common error messages.

Time-based Analysis (--time-analysis)

Groups errors into hourly and daily buckets to show when issues occur most frequently.

IP Frequency (--ips)

Extracts IPv4 addresses from log entries, counts occurrences, and shows the top sources.

Spike Detection (--spikes)

Compares error counts per time bucket against the average. Flags buckets that exceed 2x the average as potential anomalies.

Common Findings & Recommendations

• Repeated errors from the same source: Check application/service health; consider rate limiting or restart
• Time-based spikes: Correlate with cron jobs, deployments, or traffic patterns at the flagged times
• High-frequency IPs: Could indicate brute-force attempts, scrapers, or DDoS — consider firewall rules or fail2ban
• New error patterns appearing: Recent changes or deployments may have introduced regressions
• Sudden increase in warnings: Often precedes critical failures — investigate proactively

Notes

• Piped input reads from stdin; use -f - explicitly when piping
• For large log files (>100MB), consider reducing the time window with -t or pre-filtering with grep
• Time-based analysis expects standard syslog date formats (RFC 3164 or RFC 5424); custom formats may need adjustment
• IP detection uses a standard IPv4 regex — IPv6 is not currently supported
• Works without root for user-owned log files; system logs may require sudo