TL;DR A CPython dict keeps two arrays: a sparse hash table (dk_indices) holding only small integer indices, and a dense array (dk_entries) of records appended in insertion order. The layout arrived in Python 3.6 and made dicts 20–25% smaller; insertion order became part of the language spec in 3.7.

Here are two dictionaries built from the same three keys in different orders:

>>> d1 = {"a": 1, "b": 2, "c": 3}
>>> d2 = {"c": 3, "b": 2, "a": 1}
>>> d1
{'a': 1, 'b': 2, 'c': 3}
>>> d2
{'c': 3, 'b': 2, 'a': 1}
>>> d1 == d2
True

Each one prints in the order its keys went in, and iterating either gives that same order. The two are still equal, because equality compares contents and ignores order. A dict is a hash table, and a hash table scatters keys by their hash values, which have no relation to the order you added them. So the order on screen is coming from somewhere other than the hashing.

The old layout

Before CPython 3.6, a dict didn’t keep that order at all. The dict was a single sparse array of entry structs, each slot holding a hash, a key pointer, and a value pointer, roughly 24 bytes on a 64-bit build, and most of the slots sat empty, because a hash table needs slack to keep collisions rare. Iteration walked that array in slot order, which followed the hashes, so the order you got back had little to do with the order you put things in.

Two arrays

The compact dict, which landed in 3.6, splits that single array in two. The source comment in Objects/dictobject.c says: “dk_indices is actual hashtable. It holds index in entries, or DKIX_EMPTY(-1) or DKIX_DUMMY(-2).” So the hash table proper, dk_indices, no longer holds the records. It holds one small integer per slot: an index into a second array, or one of two negative markers for an empty or a deleted slot. The records live in that second array, dk_entries, a dense run of (hash, key, value) entries, and a new key is appended to the end of it.

A lookup still hashes into dk_indices to find a slot, reads the small integer sitting there, and follows it to the entry in dk_entries.

Order and memory

The order comes from that split. Because entries are appended, dk_entries holds them in the order they were inserted, so reading it from the top gives the keys back in that order, as with d1 and d2 in the opening example. The source comment says: “Since dk_entries is mostly append only, we can get insertion order by just iterating dk_entries.”

Then the memory. The empty slots are now in dk_indices, where a slot is a single byte instead of a 24-byte struct; the width steps up with the table, one byte up to 128 slots, then two, four, and eight bytes as it grows past each threshold. The full entries are packed together with no gaps between them. The 3.6 What’s New says the new dict’s memory usage is “between 20% and 25% smaller compared to Python 3.5”. The design came from a proposal by Raymond Hettinger and was first implemented in PyPy.

In 3.6 the order was there, but not guaranteed. The 3.6 What’s New says the order-preserving aspect is “considered an implementation detail and should not be relied upon”. Then in 3.7, “the insertion-order preservation nature of dict objects has been declared to be an official part of the Python language spec”.

Deletions

Deletion is why the comment says “mostly”. It does not shift the later entries down to close the gap; it sets that slot’s index in dk_indices to the DKIX_DUMMY marker and clears the record in dk_entries, leaving a hole. A later insert still appends at the end rather than filling that hole, and the holes are reclaimed only when the dict resizes and repacks dk_entries. The comment also names popitem(), which takes from the end rather than appending. So the dense array grows by appending, with a few gaps left where things were removed, and the order you iterate is the order of whatever is still in it.