# How does Claktor measure code, and why?

Claktor removes repeated logic and compresses programs to their crystallized version. Doing that reliably needs a metric. The metric is the token count of the normalized program. It measures program logic, and it is a proxy for the cost of any agent that reads the code later.

To get that count, Claktor parses each file and tokenizes a canonical form with OpenAI's open-source o200k_base vocabulary, so only the information the program carries is charged.

## What changes the count?

![Three edits to one program. Reformatting and renaming leave the count unchanged. Removing logic lowers it.](/learn/assets/count-diagram.svg)

## Why tokens?

A line count says nothing about how long each line is. A token count does. The vocabulary is o200k_base from [tiktoken](https://github.com/openai/tiktoken), and tokens are what an agent pays to read the code.

## What does the count charge for?

- Every name the code declares becomes a single `$`. Long names, short names and aliases cost the same.
- Types cost nothing. Removing an annotation or an interface moves the count by zero.
- Whitespace and comments are removed.
- Optional syntax has one form. Redundant parentheses, trailing commas and semicolon style do not count.
- Python and TypeScript are priced alike. Python's newlines, indents and dedents are charged as `;`, `{` and `}`.
- The contents of strings, regexes and docstrings are charged in full.

What is left is logic, so an agent chasing a lower number has to remove logic.

## What does a lower count look like in a diff?

The line count can stay the same while each line carries less.

```ts
// before
if (user !== null && user !== undefined && user.isActive === true) {

// after
if (user?.isActive === true) {
```

In [steipete/Markdansi](https://github.com/steipete/Markdansi) PR #23, three flush functions became one. That removed 50 production lines and about 180 tokens.

## How do you measure a change?

```sh
~/.claktor/bin/claktor length diff --compare <start commit>
```

Claktor reports the normalized tokens removed since that commit. Add `--json` for agents. Full workflow: [claktor.com/agents.md](https://claktor.com/agents.md).
