How to query the Betfair catalogue API efficiently in Java — filtering markets by event type, competition, time window, and status, caching catalogue data, and structuring queries to stay within API rate limits.
Before you can trade a market, you need to find it. The Betfair catalogue API — listMarketCatalogue and listMarketBook — is how your system discovers available markets, retrieves runner details, and checks market status. Used naively, catalogue queries are slow, expensive, and quick to hit rate limits. Used well, they’re fast and give your system everything it needs to operate efficiently.
I’ve built catalogue query layers for trading systems covering horse racing, football, and greyhounds simultaneously. This is the structure that works at production scale.
Betfair’s API enforces rate limits via a data credit system. Each API call has a weight; exceed your allocation and calls return EXCEEDED_THROTTLE. For catalogue queries:
listMarketCatalogue — 1 weight per call, up to 1000 markets per responselistMarketBook — 1 weight per call, up to 200 markets per responseThe practical implication: batch your queries. Never call listMarketCatalogue for one market at a time when you can request hundreds in a single call.
The MarketFilter object is the core of every catalogue query. Build it precisely — the more specific your filter, the smaller the response and the lower the API cost:
@Component
public class MarketFilterBuilder {
public MarketFilter horseRacingUkIreland(Duration lookAhead) {
return new MarketFilter()
.withEventTypeIds(Set.of("7")) // Horse Racing
.withMarketCountries(Set.of("GB", "IE"))
.withMarketTypeCodes(Set.of("WIN", "PLACE"))
.withMarketStartTime(new TimeRange()
.withFrom(Date.from(Instant.now()))
.withTo(Date.from(Instant.now().plus(lookAhead))))
.withInPlayOnly(false) // pre-race only
.withMarketBettingTypes(Set.of(MarketBettingType.ODDS));
}
public MarketFilter footballTopLeagues(Duration lookAhead) {
return new MarketFilter()
.withEventTypeIds(Set.of("1")) // Football
.withCompetitionIds(Set.of(
"10932509", // Premier League
"81", // Champions League
"117", // La Liga
"81", // Bundesliga
"12" // Serie A
))
.withMarketTypeCodes(Set.of("MATCH_ODDS", "BOTH_TEAMS_TO_SCORE"))
.withMarketStartTime(new TimeRange()
.withFrom(Date.from(Instant.now()))
.withTo(Date.from(Instant.now().plus(lookAhead))));
}
public MarketFilter greyhoundsUk(Duration lookAhead) {
return new MarketFilter()
.withEventTypeIds(Set.of("4339")) // Greyhound Racing
.withMarketCountries(Set.of("GB"))
.withMarketTypeCodes(Set.of("WIN"))
.withMarketStartTime(new TimeRange()
.withFrom(Date.from(Instant.now()))
.withTo(Date.from(Instant.now().plus(lookAhead))));
}
}
Always specify marketTypeCodes — without it you’ll receive every market type including exotics, forecasts, and specials that you almost certainly don’t want to trade.
listMarketCatalogue lets you specify which data you want via MarketProjection. Request only what you need — each projection adds response size and processing time:
@Service
@RequiredArgsConstructor
public class MarketCatalogueService {
private final BetfairApiClient apiClient;
private static final Set<MarketProjection> STANDARD_PROJECTIONS = Set.of(
MarketProjection.MARKET_START_TIME,
MarketProjection.RUNNER_DESCRIPTION,
MarketProjection.EVENT,
MarketProjection.COMPETITION
);
// Use when you also need runner metadata (form, silks, etc.)
private static final Set<MarketProjection> FULL_PROJECTIONS = Set.of(
MarketProjection.MARKET_START_TIME,
MarketProjection.RUNNER_DESCRIPTION,
MarketProjection.RUNNER_METADATA,
MarketProjection.EVENT,
MarketProjection.COMPETITION,
MarketProjection.MARKET_DESCRIPTION
);
public List<MarketCatalogue> getUpcomingMarkets(MarketFilter filter) {
return apiClient.listMarketCatalogue(
filter,
STANDARD_PROJECTIONS,
MarketSort.FIRST_TO_START,
1000
);
}
}
RUNNER_METADATA is expensive — it includes form data, jockey/trainer details, silks. Only request it when you genuinely need it, not on every catalogue poll.
Race card data doesn’t change frequently. Runner names, race distances, track conditions — these are stable once the market is published. Refreshing the full catalogue every poll cycle wastes API credits and adds latency.
Cache the catalogue with an appropriate TTL and only re-fetch when necessary:
@Component
@RequiredArgsConstructor
@Slf4j
public class MarketCatalogueCache {
private final MarketCatalogueService catalogueService;
private final MarketFilterBuilder filterBuilder;
private final Map<String, CachedMarket> cache = new ConcurrentHashMap<>();
private volatile Instant lastFullRefresh = Instant.EPOCH;
private static final Duration FULL_REFRESH_INTERVAL = Duration.ofMinutes(5);
private static final Duration MARKET_TTL = Duration.ofHours(1);
@Scheduled(fixedDelay = 60_000)
public void refresh() {
if (Duration.between(lastFullRefresh, Instant.now()).compareTo(FULL_REFRESH_INTERVAL) < 0) {
return; // not time for a full refresh yet
}
try {
List<MarketCatalogue> markets = catalogueService.getUpcomingMarkets(
filterBuilder.horseRacingUkIreland(Duration.ofHours(6)));
markets.forEach(m -> cache.put(m.getMarketId(),
new CachedMarket(m, Instant.now())));
// Evict expired entries
Instant evictBefore = Instant.now().minus(MARKET_TTL);
cache.entrySet().removeIf(e -> e.getValue().cachedAt().isBefore(evictBefore));
lastFullRefresh = Instant.now();
log.debug("Catalogue cache refreshed: {} markets", cache.size());
} catch (Exception e) {
log.error("Catalogue refresh failed: {}", e.getMessage());
}
}
public Optional<MarketCatalogue> getMarket(String marketId) {
CachedMarket cached = cache.get(marketId);
return cached != null ? Optional.of(cached.catalogue()) : Optional.empty();
}
public List<MarketCatalogue> getMarketsStartingWithin(Duration window) {
Instant cutoff = Instant.now().plus(window);
return cache.values().stream()
.map(CachedMarket::catalogue)
.filter(m -> m.getMarketStartTime() != null
&& m.getMarketStartTime().toInstant().isBefore(cutoff))
.sorted(Comparator.comparing(m -> m.getMarketStartTime().toInstant()))
.toList();
}
record CachedMarket(MarketCatalogue catalogue, Instant cachedAt) {}
}
The five-minute full refresh gives you an up-to-date view of the day’s markets without burning credits on every scheduling cycle. Individual markets are evicted after one hour — by which point they’ve either started or been abandoned.
Once you have the catalogue, selecting which markets to subscribe to via the Streaming API is a filtering problem. Subscribe too many markets and you overwhelm your processing pipeline; too few and you miss trading opportunities.
@Service
@RequiredArgsConstructor
public class StreamingSubscriptionManager {
private final MarketCatalogueCache catalogueCache;
private final BetfairStreamingClient streamingClient;
private static final Duration SUBSCRIBE_BEFORE_START = Duration.ofMinutes(30);
private static final Duration UNSUBSCRIBE_AFTER_start = Duration.ofMinutes(10);
@Scheduled(fixedDelay = 30_000)
public void manageSubscriptions() {
Set<String> currentSubscriptions = streamingClient.getActiveMarketIds();
// Subscribe markets starting within the window
List<String> toSubscribe = catalogueCache
.getMarketsStartingWithin(SUBSCRIBE_BEFORE_START)
.stream()
.map(MarketCatalogue::getMarketId)
.filter(id -> !currentSubscriptions.contains(id))
.toList();
// Unsubscribe markets that started more than N minutes ago
Instant startedBeforeCutoff = Instant.now().minus(UNSUBSCRIBE_AFTER_start);
List<String> toUnsubscribe = currentSubscriptions.stream()
.filter(id -> {
Optional<MarketCatalogue> market = catalogueCache.getMarket(id);
return market.map(m -> m.getMarketStartTime() != null
&& m.getMarketStartTime().toInstant().isBefore(startedBeforeCutoff))
.orElse(false);
})
.toList();
if (!toSubscribe.isEmpty()) {
streamingClient.subscribeToMarkets(toSubscribe);
log.info("Subscribed to {} new markets", toSubscribe.size());
}
if (!toUnsubscribe.isEmpty()) {
streamingClient.unsubscribeFromMarkets(toUnsubscribe);
log.info("Unsubscribed from {} finished markets", toUnsubscribe.size());
}
}
}
The subscription window of 30 minutes before start gives you enough data history to build WoM baselines and velocity trackers before the critical pre-race window. Unsubscribing shortly after start keeps the subscription list from growing unboundedly across a full racing day.
Not every market you find in the catalogue will be tradeable when the time comes. Markets can be suspended before they go live, withdrawn, or cancelled. Poll listMarketBook on your active subscription set to track status:
@Scheduled(fixedDelay = 10_000)
public void checkMarketStatuses() {
Set<String> activeIds = streamingClient.getActiveMarketIds();
if (activeIds.isEmpty()) return;
List<MarketBook> books = apiClient.listMarketBook(
new ArrayList<>(activeIds),
new PriceProjection().withPriceData(Set.of(PriceData.EX_BEST_OFFERS)),
null, null
);
for (MarketBook book : books) {
if (book.getStatus() == MarketStatus.CLOSED) {
streamingClient.unsubscribeFromMarkets(List.of(book.getMarketId()));
log.info("Market {} closed — unsubscribed", book.getMarketId());
}
}
}
Closed markets that aren’t unsubscribed continue consuming your streaming quota and memory. Clean them up promptly.
Efficient catalogue management is unglamorous but foundational. A trading system that misses markets, over-polls the API, or accumulates stale subscriptions will fail in ways that are hard to diagnose. Get the plumbing right and the strategy logic has a clean surface to work against.
If you’re building production Betfair trading systems in Java and need an engineer who has designed this infrastructure, get in touch.