package core:diff
Overview
Differences between two sequences: which tokens of a were deleted and which tokens of b were inserted.
Diffs slices of any comparable type, slices of any type with your own equality and hash, and text by lines, words, or runes. Read the result as edits, hunks, or substrings. Write it as a unified diff, in colour if wanted, with the changed parts of lines highlighted.
The Result ¶
A diff is a Changes: one bit per token of a (deleted or not) and per token of b (inserted or not).
Its size is changes_size(n, m) words, known before diffing. Everything else reads from it without allocating:
edit_iterate `Edit`s: runs of equal, deleted, and inserted tokens hunk_iterate `Hunk`s: groups of changes with equal tokens around them span_iterate the same runs as substrings of a `Text_Diff` unified_iterate the lines of a unified diff of a line diff write_unified a unified diff, written to an `io.Writer` stats the number of equal, deleted, and inserted tokens
Algorithms ¶
.Myers Myers' O(ND) diff with git's cost cap. Close to minimal. The default.
.Minimal Myers without the cap. Always minimal. Slow on very different inputs.
.Histogram git's histogram diff. Anchors on the rarest tokens, keeping moved blocks whole.
Not minimal.
.Ratcliff_Obershelp The longest common block first, then each side of it. Not minimal. Tokens occurring
in `b` more than 64 times (e.g. `}` or blank lines) extend blocks but never start
them. Hands over to Myers past a budget of work.
Options.max_cost caps Myers in edits, wherever Myers runs. For .Ratcliff_Obershelp, it is also the
budget in steps per token. 0 means the default.
Flags:
.Prune Before diffing, set aside the common prefix and suffix, tokens absent from the other
side, and runs of very frequent tokens. Often much faster.
.Compact After diffing, slide each group of changes to where git puts it: beside a change on
the other side, otherwise as far down as it goes. With `.Histogram`, merged groups
are diffed again.
.Indent_Heuristic With `.Compact`, slide each group to where the indentation and blank lines around it
read best. Line diffs of text only. Anything else is an assertion failure.
These match git exactly:
.Myers + .Prune, .Compact, and .Indent_Heuristic git diff .Minimal + .Prune, .Compact, and .Indent_Heuristic git diff --minimal .Histogram + .Compact and .Indent_Heuristic git diff --histogram
Without .Indent_Heuristic, each matches git with --no-indent-heuristic.
Memory ¶
Nothing allocates behind your back. There are two ways in:
- The
*_bufferprocedures (diff_buffer,diff_by_buffer,diff_interned_buffer, anddiff_text_buffer) never allocate. PassChangesmade fromchanges_sizewords, and scratch ofscratch_size,scratch_size_interned, orscratch_size_textbytes at any alignment. - The allocating procedures (
diff,diff_by,diff_interned, anddiff_text) take two allocators with no defaults.allocatorgives the result for you to free.transient_allocatorgives scratch that is freed before they return.
The result is allocated first and the scratch last. The scratch is freed first. Heap allocators never leak.
An allocator that frees in reverse order can be passed as both. A bump arena cannot free the scratch: it
stays until the arena is reset. Use separate arenas for the two lifetimes, or a *_buffer procedure with
scratch from a temporary region (see the examples).
Writers take scratch: []byte, not an allocator. Only .Words emphasis needs scratch:
emphasis_scratch_size bytes.
unified and write_unified_to_builder make room for their output by Sizing:
.Exact The default. Writes the diff twice and allocates exactly once. Suits any allocator.
.Grow Writes the diff once into growing memory. Close to twice as fast. Suits the heap and allocators
that resize their last allocation in place (e.g. `core:mem`'s and `core:mem/virtual`'s `Arena`).
Limits ¶
a and b together must have fewer than max(i32) tokens. Each text must be shorter than max(i32) bytes.
Going past either is an assertion failure.
Examples ¶
Example:
package main
import "core:diff"
import "core:os"
import "core:terminal"
main :: proc() {
old := "one\ntwo\nthree\n"
new := "one\n2\nthree\nfour\n"
d, err := diff.diff_text(old, new, context.allocator, context.allocator)
if err != nil {
return
}
defer diff.text_diff_destroy(d, context.allocator)
style := diff.STYLE_NONE
if terminal.color_enabled {
style = diff.STYLE_ANSI
}
_ = diff.write_unified(os.to_stream(os.stdout), d, "old.txt", "new.txt", style = style, emphasis = .Prefix_Suffix)
}
--- old.txt +++ new.txt @@ -1,3 +1,4 @@ one -two +2 three +four
Example:
// Any comparable type, read back as edits
a := []int{1, 2, 3, 4, 5}
b := []int{1, 3, 4, 6, 5}
c, err := diff.diff(a, b, context.allocator, context.allocator, opts = {algorithm = .Histogram})
if err != nil {
return
}
defer diff.changes_destroy(c, context.allocator)
it := diff.edit_iterator_make(c)
for e in diff.edit_iterate(&it) {
switch e.op {
case .Equal: fmt.println("keep ", a[e.a_lo:e.a_hi])
case .Delete: fmt.println("delete", a[e.a_lo:e.a_hi])
case .Insert: fmt.println("insert", b[e.b_lo:e.b_hi])
}
}
keep [1] delete [2] keep [3, 4] insert [6] keep [5]
Example:
// Your own equality: equal tokens must hash the same
equal_fold :: proc(x, y: string, _: rawptr) -> bool {
return strings.equal_fold(x, y)
}
hash_fold :: proc(x: string, _: rawptr) -> u64 {
h := u64(0xcbf29ce484222325)
for r in x {
h = (h ~ u64(unicode.to_lower(r))) * 0x100000001b3
}
return h
}
a := []string{"Hello", "World"}
b := []string{"hello", "there", "world"}
c, err := diff.diff_by(a, b, equal_fold, hash_fold, nil, context.allocator, context.allocator)
if err != nil {
return
}
defer diff.changes_destroy(c, context.allocator)
fmt.println(diff.stats(c))
Stats{equal = 2, deleted = 0, inserted = 1}
Example:
// Words, rendered inline
d, err := diff.diff_text("the quick brown fox", "the slow brown fox", context.allocator, context.allocator, token = .Word)
if err != nil {
return
}
defer diff.text_diff_destroy(d, context.allocator)
it := diff.span_iterator_make(d)
for s in diff.span_iterate(&it) {
switch s.op {
case .Equal: fmt.print(s.text)
case .Delete: fmt.printf("[-%s-]", s.text)
case .Insert: fmt.printf("{{+%s+}}", s.text)
}
}
the [-quick-]{+slow+} brown fox
Example:
// One arena: the result first, then the scratch in a temporary region
arena: vmem.Arena
_ = vmem.arena_init_growing(&arena)
defer vmem.arena_destroy(&arena)
arena_allocator := vmem.arena_allocator(&arena)
n := diff.token_count(old, .Line)
m := diff.token_count(new, .Line)
bits, _ := make([]u64, diff.changes_size(n, m), arena_allocator)
temp := vmem.arena_temp_begin(&arena)
scratch, _ := make([]byte, diff.scratch_size_text(n, m), arena_allocator)
d := diff.diff_text_buffer(old, new, diff.changes_from_buffer(n, m, bits), scratch)
vmem.arena_temp_end(temp)
// `d` stays valid until the arena is destroyed
Index
Types (23)
Constants (3)
Variables (0)
This section is empty.
Procedures (38)
changes_… (4)
- diff
- edits_fill
- edits_make
- emphasis_scratch_size
- hunk_iterate
- hunk_iterator_make
- is_deleted
- is_inserted
scratch_… (3)
- similarity
- span_iterate
- span_iterator_make
- stats
- text_diff_destroy
- token_count
- unified
unified_… (3)
- write_unified
- write_unified_to_builder
Procedure Groups (0)
This section is empty.
Types
Algorithm ¶
Algorithm :: enum u8 { Myers, // Myers O(ND) with a cost cap; close to minimal (default) Minimal, // Myers with no cost cap; always minimal, but can be slow on very different inputs Histogram, // anchors on rare tokens; falls back to Myers Ratcliff_Obershelp, // longest common block first; not minimal; falls back to Myers at the cost cap }
The algorithm used to compute a diff.
Changes ¶
The result of a diff: one bit per token on each side.
A set bit i on the a side means a[i] was deleted.
A set bit j on the b side means b[j] was inserted.
The a side is the first (n+63)/64 words of bits; the b side is the rest.
Related Procedures With Parameters
Related Procedures With Returns
Edit ¶
A run of tokens with the same Op: the token ranges a[a_lo:a_hi] and b[b_lo:b_hi].
A .Delete has an empty b range. An .Insert has an empty a range.
Edit_Iterator ¶
Edit_Iterator :: struct { changes: Changes, i: int, j: int, // the next token on each side a_hi: int, b_hi: int, }
Related Procedures With Returns
Emphasis ¶
Emphasis :: enum u8 { None, Prefix_Suffix, // trim the common prefix and suffix of paired deleted and inserted lines; needs no scratch Words, // word diff of each block of deleted and inserted lines; needs `emphasis_scratch_size` bytes of scratch }
How the changed part within a changed line is highlighted.
Related Procedures With Parameters
Hunk ¶
The token ranges of a group of changes and up to context_len equal tokens around it.
Option_Flag ¶
Option_Flag :: enum u8 { Prune, // pre-pass: set aside the common ends, tokens with no match, and runs of very frequent tokens Compact, // post-pass: slide each group of changes to where git puts it Indent_Heuristic, // with .Compact, for line diffs of text: slide each group by the indentation around it }
Option_Flags ¶
Option_Flags :: bit_set[Option_Flag; u8]
Options ¶
Options :: struct { algorithm: Algorithm, flags: bit_set[Option_Flag; u8], max_cost: int, }
Related Procedures With Parameters
Sizing ¶
Sizing :: enum u8 { Exact, Grow, }
How unified and write_unified_to_builder make room for the output.
.Exactwrites the diff twice: once to count its bytes, then into one allocation of exactly that size. Nothing is resized. Suits any allocator, including fixed buffers and bump arenas..Growwrites the diff once into growing memory, then shrinks it to fit. Close to twice as fast. Suits the heap and allocators that resize their last allocation in place:core:mem'sArena,Stack, andSmall_Stack, andcore:mem/virtual'sArenawhile its current block has room. With nothing else allocated from them meanwhile, these end up holding exactly the output. Inside a temporary region of eitherArena, the shrink cannot give memory back: up to as much again stays in use until the region ends. ADynamic_Arenacopies an allocation larger than its out-of-band size to grow it, and keeps every copy until it is reset. Use.Exactwith it.
Related Procedures With Parameters
Span ¶
A run of whole tokens as one substring. .Equal and .Delete spans come from a. .Insert spans come from b.
Span_Iterator ¶
Span_Iterator :: struct { d: Text_Diff, edits: Edit_Iterator, a_pos: int, b_pos: int, }
Stats ¶
Counts of tokens by what happened to them.
Style ¶
Style :: struct { file_header: string, hunk_header: string, equal: string, delete: string, insert: string, delete_emphasis: string, // the changed part within a deleted line insert_emphasis: string, // the changed part within an inserted line reset: string, }
Escape sequences written before each kind of output, and reset after it. All empty means no colour.
Related Procedures With Parameters
Related Constants
Text_Diff ¶
A diff of two strings. It owns only changes. a and b must outlive it.
Related Procedures With Parameters
Related Procedures With Returns
Text_Flag ¶
Text_Flag :: enum u8 { Ignore_CR, // "a\r\n" equals "a\n" Ignore_Trailing_Space, // whitespace at the end, before any final "\n" Ignore_All_Space, // all whitespace, apart from a final "\n" Ignore_Case, // simple per-rune case folding }
What to ignore when comparing tokens. Each flag applies to every token's text. A final "\n" is never ignored: a last line without one still differs from the same line with one.
Text_Flags ¶
Related Procedures With Parameters
Token ¶
Token :: enum u8 { Line, // up to and including "\n"; the last line may have no "\n" Word, // a run of letters, digits, and '_', a run of whitespace, or any other single rune Rune, // one rune; each byte of invalid UTF-8 is a rune of its own }
How text is split into tokens.
Related Procedures With Parameters
Unified_Iterator ¶
Unified_Iterator :: struct { d: Text_Diff, hunks: Hunk_Iterator, edits: Edit_Iterator, edit: Edit, // the edit whose lines are being given a_index: int, b_index: int, // the next line on each side a_pos: int, b_pos: int, // byte offset of that line in_hunk: bool, no_newline: bool, }
Unified_Kind ¶
Unified_Kind :: enum u8 { Hunk_Header, // "@@ -a,b +c,d @@"; the ranges are in `Unified_Line.hunk` Equal, // " line" Delete, // "-line" Insert, // "+line" No_Newline, // "\ No newline at end of file", after the line it applies to }
Unified_Line ¶
Unified_Line :: struct { kind: Unified_Kind, text: string, // the line without its prefix or newline; empty for `.Hunk_Header` and `.No_Newline` hunk: Hunk, // only set for `.Hunk_Header` a_line: int, b_line: int, }
Constants
DEFAULT_CONTEXT_LEN ¶
DEFAULT_CONTEXT_LEN :: 3
STYLE_ANSI ¶
STYLE_ANSI :: Style{file_header = ansi.CSI + ansi.BOLD + ansi.SGR, hunk_header = ansi.CSI + ansi.FG_CYAN + ansi.SGR, delete = ansi.CSI + ansi.FG_RED + ansi.SGR, insert = ansi.CSI + ansi.FG_GREEN + ansi.SGR, delete_emphasis = ansi.CSI + ansi.FG_RED + ";" + ansi.INVERT + ansi.SGR, insert_emphasis = ansi.CSI + ansi.FG_GREEN + ";" + ansi.INVERT + ansi.SGR, reset = ansi.CSI + ansi.RESET + ansi.SGR}
Colours similar to git's defaults.
STYLE_NONE ¶
STYLE_NONE :: Style{}
Variables
This section is empty.
Procedures
changes_destroy ¶
changes_destroy :: proc(c: Changes, allocator: runtime.Allocator, loc := #caller_location) {…}
Frees c with the allocator it was made with.
changes_from_buffer ¶
@(require_results) changes_from_buffer :: proc(n, m: int, bits: []u64, loc := #caller_location) -> Changes {…}
Makes Changes from a caller-provided buffer of at least changes_size(n, m) words.
changes_make ¶
@(require_results) changes_make :: proc( n, m: int, allocator: runtime.Allocator, loc := #caller_location, ) -> (c: Changes, err: runtime.Allocator_Error) {…}
changes_size ¶
Returns the number of words Changes needs for inputs of length n and m.
diff ¶
@(require_results) diff :: proc( a, b: []$T, allocator: runtime.Allocator, transient_allocator: runtime.Allocator, opts: Options = Options{}, loc := #caller_location, ) -> (c: Changes, err: runtime.Allocator_Error) {…}
diff_buffer ¶
diff_buffer :: proc( a, b: []$T, changes: Changes, scratch: []u8, opts: Options = Options{}, loc := #caller_location, ) {…}
diff_by ¶
@(require_results) diff_by :: proc( a, b: []$T, equal: proc(x, y: $T, user_data: rawptr) -> bool, hash: proc(x: $T, user_data: rawptr) -> u64, user_data: rawptr, allocator: runtime.Allocator, transient_allocator: runtime.Allocator, opts: Options = Options{}, loc := #caller_location, ) -> (c: Changes, err: runtime.Allocator_Error) {…}
diff_by_buffer ¶
diff_by_buffer :: proc( a, b: []$T, equal: proc(x, y: $T, user_data: rawptr) -> bool, hash: proc(x: $T, user_data: rawptr) -> u64, user_data: rawptr, changes: Changes, scratch: []u8, opts: Options = Options{}, loc := #caller_location, ) {…}
equal and hash must agree: equal tokens must hash the same.
diff_interned ¶
@(require_results) diff_interned :: proc( a, b: []i32, id_count: int, allocator: runtime.Allocator, transient_allocator: runtime.Allocator, opts: Options = Options{}, loc := #caller_location, ) -> (c: Changes, err: runtime.Allocator_Error) {…}
diff_interned_buffer ¶
diff_interned_buffer :: proc( a, b: []i32, id_count: int, changes: Changes, scratch: []u8, opts: Options = Options{}, loc := #caller_location, ) {…}
Diffs already-interned inputs: equal tokens share an ID, and every ID is in [0, id_count).
diff_text ¶
@(require_results) diff_text :: proc( a, b: string, allocator: runtime.Allocator, transient_allocator: runtime.Allocator, token: Token = Token.Line, flags: bit_set[Text_Flag; u8] = Text_Flags{}, opts: Options = Options{}, loc := #caller_location, ) -> (d: Text_Diff, err: runtime.Allocator_Error) {…}
diff_text_buffer ¶
@(require_results) diff_text_buffer :: proc( a, b: string, changes: Changes, scratch: []u8, token: Token = Token.Line, flags: bit_set[Text_Flag; u8] = Text_Flags{}, opts: Options = Options{}, loc := #caller_location, ) -> Text_Diff {…}
Diffs a and b into changes, sized for token_count(a, token) and token_count(b, token). Never allocates.
edit_iterate ¶
@(require_results) edit_iterate :: proc "contextless" (it: ^Edit_Iterator) -> (e: Edit, ok: bool) {…}
Iterates over edits in order. Never allocates.
Usage:
it := diff.edit_iterator_make(changes)
for e in diff.edit_iterate(&it) {
...
}
edit_iterator_make ¶
@(require_results) edit_iterator_make :: proc "contextless" (c: Changes) -> Edit_Iterator {…}
Makes an iterator over every edit in c.
edit_iterator_make_hunk ¶
@(require_results) edit_iterator_make_hunk :: proc "contextless" (c: Changes, h: Hunk) -> Edit_Iterator {…}
Makes an iterator over the edits inside h, including its context.
edits_fill ¶
@(require_results) edits_fill :: proc(c: Changes, buf: []Edit, loc := #caller_location) -> []Edit {…}
Fills buf with every edit in c and returns the filled part. buf must hold at least edit_count(c) edits.
edits_make ¶
@(require_results) edits_make :: proc( c: Changes, allocator: runtime.Allocator, loc := #caller_location, ) -> (edits: []Edit, err: runtime.Allocator_Error) {…}
Allocates exactly edit_count(c) edits and fills them.
emphasis_scratch_size ¶
@(require_results) emphasis_scratch_size :: proc( d: Text_Diff, emphasis: Emphasis, loc := #caller_location, ) -> int {…}
Returns the scratch bytes write_unified, unified_size, and unified need for emphasis. 0 unless it is .Words.
hunk_iterate ¶
@(require_results) hunk_iterate :: proc "contextless" (it: ^Hunk_Iterator) -> (h: Hunk, ok: bool) {…}
Iterates over hunks in order. Hunks whose context would overlap or touch are merged. Never allocates.
Usage:
hunks := diff.hunk_iterator_make(changes)
for h in diff.hunk_iterate(&hunks) {
edits := diff.edit_iterator_make_hunk(changes, h)
for e in diff.edit_iterate(&edits) {
...
}
}
hunk_iterator_make ¶
@(require_results) hunk_iterator_make :: proc "contextless" ( c: Changes, context_len: int = DEFAULT_CONTEXT_LEN, ) -> Hunk_Iterator {…}
A negative context_len is treated as 0.
scratch_size ¶
@(require_results) scratch_size :: proc "contextless" (n, m: int, opts: Options = Options{}) -> int {…}
Returns the scratch bytes diff_buffer and diff_by_buffer need for inputs of length n and m.
The token IDs last the whole diff. The interning table and the algorithm share the rest, one after the other.
scratch_size_interned ¶
@(require_results) scratch_size_interned :: proc "contextless" ( n, m: int, id_count: int, opts: Options = Options{}, ) -> int {…}
Returns the scratch bytes diff_interned_buffer needs for already-interned inputs.
scratch_size_text ¶
@(require_results) scratch_size_text :: proc "contextless" (n, m: int, opts: Options = Options{}) -> int {…}
Returns the scratch bytes diff_text_buffer needs for n and m tokens.
The token IDs, and each line's indent with .Indent_Heuristic, last the whole diff. The token starts and
the interning table share the rest with the algorithm, one after the other.
similarity ¶
Returns 2*equal / (n+m), from 0 (nothing in common) to 1 (identical); 1 when both inputs are empty.
span_iterate ¶
@(require_results) span_iterate :: proc(it: ^Span_Iterator) -> (s: Span, ok: bool) {…}
Iterates over spans in order. Never allocates.
Usage:
it := diff.span_iterator_make(d)
for s in diff.span_iterate(&it) {
...
}
span_iterator_make ¶
@(require_results) span_iterator_make :: proc "contextless" (d: Text_Diff) -> Span_Iterator {…}
text_diff_destroy ¶
text_diff_destroy :: proc(d: Text_Diff, allocator: runtime.Allocator, loc := #caller_location) {…}
Frees d with the allocator it was made with. The texts it refers to are not its own.
token_count ¶
Returns the number of tokens in s: the n or m used to size Changes and scratch.
unified ¶
@(require_results) unified :: proc( d: Text_Diff, allocator: runtime.Allocator, a_name: string = "a", b_name: string = "b", context_len: int = DEFAULT_CONTEXT_LEN, style: Style = STYLE_NONE, emphasis: Emphasis = Emphasis.None, scratch: []u8 = nil, sizing: Sizing = Sizing.Exact, loc := #caller_location, ) -> (s: string, err: runtime.Allocator_Error) {…}
Returns d as a unified diff, allocated from allocator. Free it with the same allocator.
.Exact makes one allocation of exactly unified_size bytes. .Grow grows the memory while writing,
then shrinks it to fit. On failure, nothing stays allocated.
unified_iterate ¶
@(require_results) unified_iterate :: proc(it: ^Unified_Iterator) -> (line: Unified_Line, ok: bool) {…}
Iterates over the lines of unified output in order. Never allocates.
Usage:
it := diff.unified_iterator_make(d)
for line in diff.unified_iterate(&it) {
switch line.kind {
case .Hunk_Header: ...
case .Equal: ...
case .Delete: ...
case .Insert: ...
case .No_Newline: ...
}
}
unified_iterator_make ¶
@(require_results) unified_iterator_make :: proc( d: Text_Diff, context_len: int = DEFAULT_CONTEXT_LEN, loc := #caller_location, ) -> Unified_Iterator {…}
Makes an iterator over the lines of unified output, without the "---" and "+++" file headers. d must be a line diff.
unified_size ¶
@(require_results) unified_size :: proc( d: Text_Diff, a_name: string = "a", b_name: string = "b", context_len: int = DEFAULT_CONTEXT_LEN, style: Style = STYLE_NONE, emphasis: Emphasis = Emphasis.None, scratch: []u8 = nil, loc := #caller_location, ) -> int {…}
Returns the exact number of bytes write_unified would write.
write_unified ¶
write_unified :: proc( w: io.Stream, d: Text_Diff, a_name: string = "a", b_name: string = "b", context_len: int = DEFAULT_CONTEXT_LEN, style: Style = STYLE_NONE, emphasis: Emphasis = Emphasis.None, scratch: []u8 = nil, loc := #caller_location, ) -> io.Error {…}
Writes d as a unified diff, file headers included. Writes nothing without changes. d must be a line diff. Never allocates.
write_unified_to_builder ¶
write_unified_to_builder :: proc( sb: ^strings.Builder, d: Text_Diff, a_name: string = "a", b_name: string = "b", context_len: int = DEFAULT_CONTEXT_LEN, style: Style = STYLE_NONE, emphasis: Emphasis = Emphasis.None, scratch: []u8 = nil, sizing: Sizing = Sizing.Exact, loc := #caller_location, ) -> runtime.Allocator_Error {…}
Appends d to sb as a unified diff, file headers included. d must be a line diff.
.Exact grows sb once, by exactly unified_size bytes. .Grow grows it while writing.
If sb cannot grow enough, nothing is appended.
Procedure Groups
This section is empty.
Source Files
- compact.odin
- diff.odin
- doc.odin
- edit.odin
- emphasis.odin
- histogram.odin
- intern.odin
- myers.odin
- prune.odin
- ratcliff_obershelp.odin
- scratch.odin
- text.odin
- unified.odin
Generation Information
Generated with odin version dev-2026-10 (vendor "odin") Windows_amd64 @ 2026-10-18 22:42:50.976285200 +0000 UTC