Pagination of large data volumes

You are a backend engineer expert in data APIs. Help me with pagination: the scenario is [DESCRIBE: the listing — what it lists, total volume, necessary ordering, filters —, who consumes: frontend with infinite scroll, table with numbered pages, export, another API], stack [STACK + DATABASE], and the pain is [PAIN: OFFSET getting slow on deep pages, duplicate/skipped items when the list changes, need 'go to page 47', volume stalling everything]. Deliver: the method choice with the physics explained (offset/limit — simple with two sins: the database READS and discards everything before the offset — page 1000 that scans 50k rows, and drift — the item inserted/deleted in the middle shifts the window: duplicate or skipped in user scroll; cursor/keyset — the right standard for volume and infinite scroll: 'give me the next N after THIS point' using the ordered column — stable and equally fast on page 1 and 10,000; the verdict for my case — and the honest hybrid when the product DEMANDS numbered pages: offset with max depth limit), keyset implementation done right in my stack (cursor over single column × mandatory tiebreaker when ordering has duplicates — the pair (created_at, id) with tuple comparison in WHERE: the complete SQL/ORM example; opaque encoded cursor — base64 of the pointer, not raw ID exposed: the client doesn't craft cursor by hand; descending ordering and 'previous' handled), the index that makes it all work (composite index matching EXACTLY the ORDER BY + WHERE — without it, keyset becomes disguised offset: the EXPLAIN checked), the clean API contract (response format: data + next_cursor (null at end) + has_more — and the limit with server-enforced ceiling: client doesn't ask for 100k; filters that participate in the cursor — changing filter invalidates cursor: the behavior defined), neighboring cases solved (the total count that costs a lot — the honest decision: exact when cheap, estimated or omitted when not; large export that isn't screen pagination — streaming/async job; infinite scroll on frontend consuming correctly — the example), and if you paste your current query/endpoint: the migration from offset to keyset step-by-step with compatibility. Objective: page 10,000 as fast as the first — and no duplicate items in the user's scroll.