Note: This is a part of our series on API Pagination where we solve common developer queries in detail with common examples and code snippets. Please read the full guide here where we discuss page size, error handling, pagination stability, caching strategies and more.
Page size — the number of records returned per API request - is one of the most consequential configuration decisions in a paginated API. Too small, and consumers make hundreds of unnecessary requests to retrieve a full dataset. Too large, and you risk timeout errors, memory pressure on your server, and slow response times that break client-side rendering.
There is no universal right answer, but there are clear frameworks for finding the right answer for your specific case.
What "Page Size" Means
In a paginated API, page size (also called limit, per_page, or size depending on the API) controls how many records are returned per request. The consumer increments a page or cursor to retrieve subsequent batches.
GET /employees?page=1&per_page=50
GET /employees?page=2&per_page=50Your job as the API designer is to pick a sensible default, enforce a safe maximum, and let consumers override the default within that ceiling.
1. Understand the data characteristics
The size and structure of individual records is the first variable to nail down.
Small, flat records (IDs, names, status fields — 1–5 KB each): you can safely return 100–200 per page without straining response payload sizes.
Typical business records (employee profiles, CRM contacts, support tickets — 5–20 KB each, with some nesting): 25–100 per page is the practical range for most APIs.
Complex or deeply nested records (job applications with embedded assessments, financial transactions with line items — 20–50+ KB each): keep page size at 10–25 to avoid response payloads exceeding 1–2 MB.
Media metadata or documents (large embedded blobs, rich-text fields): 5–20 per page, and consider whether the heavy fields should be excluded from list endpoints entirely and fetched only on individual record calls.
A fast check: multiply your average record size by your intended page size. If the math produces a payload above 2 MB, reduce the page size.
2. Factor In Network Latency and Bandwidth
Network conditions vary significantly across your consumer base:
- Browser clients on average connections: target response times under 500ms. Test your API under realistic page sizes and check P95 response times — if a page of 100 records takes 800ms on your test environment, your production P95 will likely be worse.
- Mobile clients on cellular: smaller pages (25–50) improve perceived responsiveness and reduce the impact of dropped connections mid-response.
- Server-to-server batch sync jobs: no user is waiting for each response, so larger page sizes (200–500) are appropriate to minimize the number of round trips for large dataset pulls.
3. Evaluate Server-Side Performance
Larger pages put more load on your database and API server per request. Key considerations:
Database query cost: a LIMIT 500 query scans and returns 10x more rows than LIMIT 50. For indexed queries on normalized tables this is often acceptable; for complex joins or aggregations, the cost multiplies fast.
Memory allocation: each in-flight large-page request holds the full result set in memory until serialization completes. Under concurrent load, this can spike memory usage substantially.
Timeout risk: if a consumer requests a very large page on a slow query, the request may time out partway through. Set your max page size conservatively and enforce it server-side - do not trust the consumer to be reasonable.
Test at realistic data volumes, not dev-environment datasets with 500 rows. A query that runs in 50ms against 500 rows may take 4 seconds against 5 million.
4. Consider the Consumer Experience
Always allow consumers to specify page size. A fixed page size optimized for browser pagination is wrong for a batch sync job, and vice versa. Expose a parameter:
GET /contacts?page=2&per_page=100Enforce a ceiling. Even with consumer control, set a maximum the server will honor. A request for per_page=10000 should either be rejected with a 400 or silently capped at your maximum.
Return pagination metadata. Consumers should not have to guess whether there are more pages:
{
"data": [ ... ],
"pagination": {
"page": 2,
"per_page": 50,
"total_records": 1247,
"total_pages": 25,
"next_page": 3,
"next_cursor": "eyJpZCI6MTAwfQ"
}
}total_records and total_pages let consumers pre-allocate storage and report progress. next_cursor supports cursor-based consumers even alongside page-based navigation.
5. Recommended Page Size Ranges by Use Case
Based on common API design patterns and real-world benchmarks:
How major APIs set page size in practice:
- GitHub REST API: default 30, maximum 100
- Stripe: default 10, maximum 100
- HubSpot: maximum 100 per endpoint (varies by object type)
- Salesforce: default 2000 maximum (very high - Salesforce optimizes for bulk data access)
- Jira: default 50, maximum 100
- Zendesk: default 100, maximum 100 (fixed)
The range across real-world APIs is wide. Your defaults should reflect your specific query patterns, not what another API does.
6. Test Before You Fix a Default
Don't pick a page size based on intuition alone. Before shipping:
- Load test your API at multiple page sizes (10, 25, 50, 100, 200) under concurrent request load
- Measure P95 response time at each size — not just average
- Monitor server memory during the test; look for spikes under simultaneous large-page requests
- Ask early consumers what sizes they're actually requesting — they will tell you where the pain is
A common finding: developers set a conservative default of 25 and discover their largest consumer is using ?per_page=25& in a loop making 400 requests to sync 10,000 records. A default of 200 with a max of 1000 would have served them better.
When You're Consuming (Not Building) a Paginated API
If you are integrating with third-party APIs — HRIS platforms, CRMs, ATS systems, accounting software — you are on the other side of the equation. You do not control the page size. You adapt to whatever the upstream API enforces.
The problem: every platform is different.
- BambooHR might cap at 50 records per page
- Workday might allow 200
- Salesforce might return 2000
- Each platform uses different parameter names (
page,offset,cursor,pageToken,next) - Some platforms don't return a total count, so you don't know how many pages to expect
When you are building integrations across multiple platforms, you end up maintaining separate pagination logic for each - correctly handling each platform's parameters, response format, and edge cases (missing total counts, inconsistent last-page detection, cursor invalidation).
Knit's unified API abstracts this. When your application calls Knit to fetch employee data, Knit handles pagination internally against whatever HRIS platform the customer has connected — Workday, BambooHR, Darwinbox, ADP, HiBob, and 150+ others. Your code makes a single normalized request; Knit handles per-platform pagination and returns a complete, consistent dataset.
FAQs
What is a good default page size for a REST API?
Knit's unified API uses 25–100 records per page as a default for most business data endpoints — a range that works well for typical employee, contact, or ticket records. For your own API, 25–50 is a safe starting default for business records: small enough to keep response times under 500ms in most configurations, large enough to be useful for consumers who need moderate data volumes. Adjust up or down based on your actual record sizes and database query performance at those sizes.
What is the difference between limit/offset and cursor-based pagination?
Limit/offset pagination uses numeric page and size parameters (?page=2&per_page=50) and is easy to implement and understand. Its weakness: if records are added or deleted between requests, the offset shifts and consumers may see duplicate or skipped records. Cursor-based pagination returns an opaque token pointing to the position after the last returned record; the next request passes the token instead of a page number. Cursors are stable under inserts and deletes and are the standard choice for real-time feeds or frequently updated datasets. Knit uses cursor-based pagination internally when connecting to platforms that support it, falling back to offset where cursors are unavailable.
Should I let API consumers set their own page size?
Yes — always allow consumers to specify a page size via a parameter. Different consumers have legitimately different needs: a batch sync job benefits from large pages (200–500), while a UI component loading records for display benefits from smaller pages (10–25). Enforcing a fixed page size that fits your average case will be wrong for your edge cases. Set a maximum ceiling the server enforces, and document both the default and the maximum clearly.
What happens if I set my API page size too large?
Large page sizes increase response payload size, server memory allocation per request, and database query execution time. Under concurrent load, multiple simultaneous large-page requests can spike memory usage and trigger timeout errors. Consumers who receive very large responses may hit client-side memory limits or parsing timeouts, especially in mobile environments. If your API has no max ceiling, a single malicious or misconfigured consumer can send ?per_page=100000 and effectively run a denial-of-service attack against your database. Always enforce a server-side maximum.
How do I paginate through all records in a REST API?
The standard pattern: start at page 1, request your target page size, and loop until the response contains fewer records than the page size (or a next cursor/link is absent). In Python:
all_records = []
page = 1
per_page = 100
while True:
response = requests.get(
"https://api.example.com/employees",
params={"page": page, "per_page": per_page},
headers={"Authorization": f"Bearer {token}"}
)
batch = response.json().get("data", [])
all_records.extend(batch)
if len(batch) < per_page:
break # Last page reached
page += 1When integrating with multiple third-party platforms, Knit handles this pagination loop for you — your application calls a single Knit endpoint and receives the full, normalized dataset without implementing per-platform pagination logic.
What is cursor-based pagination and when should I use it?
Cursor-based pagination replaces the page number with a pointer (cursor or token) to the position after the last returned record. Instead of ?page=3&per_page=50, the consumer sends ?cursor=eyJpZCI6MTUwfQ&per_page=50. The server returns the next batch starting after that position. Use cursor-based pagination when your dataset is updated frequently (new records inserted, existing records deleted) — offset-based pagination is unstable under these conditions and will produce duplicate or missed records between page requests. For static or infrequently updated datasets, offset is simpler and perfectly adequate.
How do major APIs like GitHub and Stripe set their page sizes?
GitHub's REST API defaults to 30 records per page with a maximum of 100. Stripe defaults to 10 with a maximum of 100. Zendesk fixes page size at 100 with no consumer override. Salesforce allows up to 2000 records per query — tuned for bulk data access rather than interactive pagination. HubSpot caps at 100 per endpoint. The wide variance reflects each platform's data model, typical use case, and database architecture. When you integrate with multiple of these APIs (as most B2B products do), you need pagination logic customized for each. Knit normalizes this across 150+ platforms so your integration code handles none of it directly.
How does Knit handle pagination when fetching data from third-party HRIS or CRM platforms?
Knit handles all pagination internally against the upstream SaaS platform — including per-platform page size limits, cursor management, offset handling, and last-page detection. When your application calls Knit's unified API to fetch employee records from Workday, BambooHR, or any of 150+ connected platforms, Knit iterates through all pages of the upstream API response and returns the complete, normalized dataset. Your integration code does not need to implement or maintain platform-specific pagination logic.
Try Knit for Third-Party API Integrations
If your application integrates with HRIS, CRM, ATS, or accounting platforms, Knit handles pagination, authentication, rate limiting, and data normalization across 150+ business apps — so you do not manage any of it per-platform.
