JSONL Dedupe
Remove duplicate JSON objects from NDJSON or JSONL input by full record or selected key fields.
O
M
T
Deduplicate JSON Lines records
Remove duplicate objects from NDJSON or JSONL logs. Match on the full serialized record or on selected top-level key fields, and keep the first or last occurrence.
Related: JSON to JSON Lines and CSV Dedupe.
Frequently Asked Questions
Are blank lines allowed?
Blank lines are ignored. Each non-empty line must be a valid JSON object.