Citation import benchmark: reproducible metadata checks
Download Lumina’s synthetic NBIB and RIS fixtures and reproduce record counts, missing-abstract checks, migration limits and cross-database deduplication.
Quick answer
This benchmark checks five downloadable synthetic citation files with Lumina’s production parsers and duplicate matcher. It tests record counts, title and abstract preservation, DOI mapping and one deliberate duplicate. It does not measure screening speed, clinical recall or real-world import accuracy.
Dataset and expected outcomes
Fixture set v1 is released as CC0. The titles, authors, journal, source identifiers and DOI-shaped strings are synthetic and must not be cited as research. No API calls or research data are required.
NBIB and full-record RIS each contain two corresponding records with abstracts. The missing-abstract RIS has one title and no abstract. The Rayyan-style RIS has one citation with unsupported note tags. The overlap RIS contains one duplicate and one additional record.
Run the actual implementation
From a Lumina checkout with its Python dependencies installed, run: python manage.py citation_import_benchmark. Add --output /tmp/citation-results.json to save JSON. The command uses parse_content and find_duplicate_in_project, with unsaved Paper objects.
The command asserts matching titles, abstracts, authors, dates, journals, DOIs and source identifiers between the paired NBIB/RIS inputs. It reports missing abstracts and verifies that the overlap produces four input records, one duplicate and three unique citations.
Interpret the results
Passing these examples means those controlled cases behave as documented. The set is small, deliberately constructed, and not a random sample of database exports. It cannot support a percentage accuracy claim or a comparison with competing tools.
A production evaluation should predefine a representative corpus, expected field values, difficult duplicate pairs and false-positive cases. Publish corpus provenance, license, software revision, dependency versions and both failed and successful cases.
Editorial and product review
Prepared by the Lumina Editorial Team as a product implementation check on October 7, 2026. No independent scientific reviewer is claimed. The downloadable report records the source revision, hashes and dependency versions used in the run.
Send corrections through our contact page. Methodological guidance on this site cites primary sources; synthetic product examples are explicitly labeled and are separate from scientific validation.
| Input | Records | With abstracts | Check |
|---|---|---|---|
| pubmed-example.nbib | 2 | 2 | Matches the full-record RIS fields |
| full-records.ris | 2 | 2 | Paired citation metadata preserved |
| missing-abstract.ris | 1 | 0 | Missing abstract remains visible |
| rayyan-style-example.ris | 1 | 1 | Citation only; note tags not mapped |
| embase-overlap.ris | 2 | 2 | One duplicate with NBIB input |
Download and inspect the examples
These CC0 examples are synthetic. Their identifiers do not refer to real publications. Use them to check an import, not as research evidence.
Read the benchmark method and limitations →Sources and editorial review
Maintained by the Lumina Editorial Team. Product mappings and examples were checked against Lumina’s implementation on October 7, 2026. External interfaces can change; consult the primary documentation below. This is an independent product guide, without a claim of independent scientific review.
Suggest a correctionInspect the screening workflow
Explore the public demo with sample papers. AI recommendations remain advisory; reviewers make the screening decisions.