grep or rg: Command combinations for locating errors in large log files

Conclusion first: It is not an either-or choice

When troubleshooting logs on a server, tool choice depends on three things: whether the environment allows installing rg, whether the logs are a single large file or a directory, and whether continuous tracking is needed. grep's advantage is that it is almost everywhere and suitable for pipelines, scripts, old systems, and read-only environments; rg's advantage is that recursive searches are parallel by default, default output is more readable, and it has its own set of ignore rules. That is also where the problem lies: rg skips hidden files, files excluded by ignore rules, and binary files by default, so replacing grep -R with it directly may miss logs. Narrowing the scope before searching large logs is more important than switching tools.

Scope: The following commands use the common behavior of GNU grep and ripgrep as examples; options may differ across distributions, BusyBox grep, and BSD grep, so verify with grep --help and rg --help before using them in production.

Scenario 1: A single large file, find only error lines

First clarify the log level field. Do not search only for lowercase error, because many logs also contain error_code. Choose the pattern based on the actual format first:

# grep：行号 + 文本模式，适合容器/老服务器
grep -n -a -E 'ERROR|Exception|FATAL|panic' /var/log/app/app.log

# rg：同样按行输出，默认会显示行号；这里显式写出 -n
rg -n -a -e 'ERROR|Exception|FATAL|panic' /var/log/app/app.log

-a treats the file as text. When NUL bytes occasionally appear in logs, grep may only report Binary file matches, and rg may skip the file directly; adding -a avoids this situation where it looks like there are no results. If the log is definitely plain text, you can omit it.

If you only want to see the first few matches, do not truncate context groups directly with | head; prefer limiting the number of matches:

grep -m 50 -n -a -E 'ERROR|Exception|FATAL|panic' /var/log/app/app.log
rg -m 50 -n -a -e 'ERROR|Exception|FATAL|panic' /var/log/app/app.log

-m is the maximum number of matching lines output per file; implementation details may differ, but it is sufficient for quick sampling. When you need a fixed string instead of a regex, use -F for grep and also -F for rg; when you need to ignore case, both use -i. For example:

grep -inF 'connection refused' /var/log/app/app.log
rg -inF 'connection refused' /var/log/app/app.log

Scenario 2: Preserve surrounding context

Troubleshooting an error usually requires more than one line. Both support -A (after), -B (before), and -C (before and after):

grep -n -a -C 3 -E 'ERROR|Exception|FATAL|panic' /var/log/app/app.log
rg -n -a -C 3 -e 'ERROR|Exception|FATAL|panic' /var/log/app/app.log

When there is a lot of output, view it paginated and preserve colors:

grep -n -a --color=always -C 3 -E 'ERROR|Exception|FATAL|panic' /var/log/app/app.log | less -R
rg -n -a --color=always -C 3 -e 'ERROR|Exception|FATAL|panic' /var/log/app/app.log | less -R

--color=always is used so colors are retained after passing through a pipe, and less -R interprets the color control characters. If you only want to see matching lines and file names, you do not need colors; just remove it. When parsing in scripts, do not rely on colors and group headings; use --color=never for grep and --no-heading --color=never for rg.

Scenario 3: Recursive search in a directory, excluding archive and dependency directories

Log directories often contain archives, compressed packages, application code, or dependency directories. Recursive search with grep requires explicitly limiting file types and excluding directories:

grep -r -n -a --include='*.log' --exclude-dir=archive --exclude-dir=node_modules --exclude-dir=.git -E 'ERROR|Exception|FATAL|panic' /var/log/app

Here use -r instead of -R: -r recurses into directories, while -R follows all symbolic links. If the log directory has symbolic links pointing elsewhere, first confirm whether they should be followed. --include='*.log' includes only log files, avoiding pulling in compressed packages and binaries. For multiple excluded directories, write multiple --exclude-dir options; this is more reliable than putting braces inside quotes. Compressed logs such as .gz are not decompressed directly by ordinary grep or rg; you need to decompress them first or use a tool such as zgrep.

rg is recursive by default and by default respects .gitignore and skips hidden files and binary files. For log directories, this default behavior may do more harm than good:

rg -n -a --no-ignore --hidden -g '*.log' -g '!archive/**' -g '!node_modules/**' -g '!.git/**' -e 'ERROR|Exception|FATAL|panic' /var/log/app

--no-ignore prevents rg from skipping paths because of ignore files, and --hidden lets hidden directories enter the search; once hidden files are enabled, use rules such as -g '!.git/**' to exclude unrelated directories. -g '*.log' limits the search scope to .log files. If log file names do not have a .log suffix, do not add this glob, or change it to the actual suffix. Do not treat -uuu as a universal speed-up switch: it disables ignore rules, hidden file filtering, and binary filtering at the same time, which easily brings in unrelated files and makes output harder to read.

Scenario 4: Use a streaming pipeline for logs being written

Searching a large file that is being appended to directly only shows content already written at the time of the search; it does not continuously track it. To see new errors in real time, pipe tail -f into grep or rg:

tail -f /var/log/app/app.log | grep --line-buffered -n -a -E 'ERROR|Exception|FATAL|panic'
tail -f /var/log/app/app.log | rg --line-buffered -n -a -e 'ERROR|Exception|FATAL|panic'

--line-buffered flushes output line by line; otherwise pipe buffering may make it seem like there is no movement. Note that the line numbers here are relative to the tail output stream, not absolute line numbers in the original file. If you need original file line numbers, search the file directly, or use a log tool that preserves line numbers; do not compare streaming line numbers with editor line numbers.

If logs have already rotated, first confirm the time ranges of the current file and archived files, then decide which one to search. Do not start by recursively searching all of /var/log; first narrow down the service directory and file name, which can significantly reduce disk reads. This advice applies to both grep and rg.

A comparison table

The ... in the table represents the patterns and file paths from the previous text; replace them with complete arguments when actually running the commands.

How to choose: by environment, not by preference

If the target machine is a production container, a minimal system, an old distribution, or you can only run read-only commands through a jump host, prefer grep. It requires no additional installation, has stable pipeline behavior, and is suitable for one-off troubleshooting scripts. If the log directory has many levels and many files, and rg is already installed on the machine, you can use rg for recursive filtering first; however, in log directories, explicitly handle --hidden, --no-ignore, and -a, otherwise the default rules may make results incomplete.

For a single large file, rg is not necessarily much faster than grep: parallel search depends more on the number of files than on the size of one file; both must read the file to the end. What really saves time is first narrowing the time window, limiting file suffixes, excluding archive directories, and then limiting the number of output lines. Replacing grep ERROR /var/log with rg ERROR /var/log is not an optimization; it is just switching to a tool with more default rules.

A reusable troubleshooting order is: first use ls -lh and tail -n to confirm the log size and latest content; then use grep or rg with -n and -a to find error lines; add -C 3 when you need the causal chain; when searching directories, add file type and exclusion rules; when observing in real time, use tail -f with --line-buffered. Tools can be replaced, but do not skip this order.