/ˈkeɪtlɪn ˈsiːzɚ/

AI Language Engineer & Digital Tinkerer they / them

Hello — I’m Caitlin Cisar, a Language Engineer working at Amazon on LLM evaluation frameworks and human language technology.

I have a particular interest in large language models (LLMs) and LLM-as-a-Judge (LLMaaJ) systems. I believe we’ve entered an era where safe artificial intelligence includes understanding how humans and machines use language, thoughtful data practices, and the mindset that the future is always scientifically worthwhile.

Before starting this journey, I was a professional photographer for over a decade. When I’m not toiling away at linguistics or LLMs, I’m likely taking film photos, hiking, writing, gaming, or spending time with my lovely family.

See more of my research for notable projects or visit my gallery to see my photography.

Skillset

⬡

NLP & NLU

Developing linguistic analysis pipelines and applying linguistic theory across production grade NLP systems.

◈

LLM & VLM Benchmarks

Building benchmarks for frontier LLMs and VLMs across long-context and multi-turn dialogue with turn-level failure localization.

⊨

LLM-as-a-Judge (LLMaaJ)

Designing and calibrating LLMaaJ systems with multi-panel judge loops and controlling for accuracy with human-in-the-loop feedback.

⋈

Annotation & Data Quality

Improving annotation pipelines and UIs, guideline authoring, and demystifying inter-annotator agreement metrics.

⊹

Python & Tooling

Coding from-scratch JS websites and dashboards, phonaesthetics generators, and standalone HTML-based tools.

⟁

Synthetic Data

Generating tens of thousands of synthetic examples across dozens of use cases for experimental and research studies.

Timeline

2026 — Present
AI Language Engineer

Amazon

Working on LLMaaJ frameworks and model evaluation.
2025 — 2026
Language Data Scientist

Innodata

Built long-context and multi-turn benchmarks, LLM-as-a-Judge (LLMaaJ) systems, and crafted tools for large-scale MLLMs evaluation.
2022 — 2024
M.A. Linguistics +
Human Language Technology Certificate

University of Colorado Boulder

Graduate work in computational linguistics, with projects spanning LLM-generated text detection and corpora research on gendered vocatives.
2021 — 2025
ML Data Linguist

Amazon Web Services

Four years working on data and quality frameworks for production ML, plus co-leading research on model steerability.
2018 — 2021
B.A. Linguistics + Philosophy Minor

Iowa State University

The formal start of my linguistics and philosophy studies. Topics covered computational linguistics, language and gender, and analytical philosophy.
2012 — 2023
Professional Photographer

Central Iowa

Spent a decade as a (digital) photographer, where I specialized in portraiture, weddings, and cats.
2000s — Present
Digital Tinkerer

The Internet

Picked up HTML/CSS by being chronically online, learned photo-editing programs like PaintShop Pro, and played a lot of Neopets.