| Title: | High-Performance GPS to GTFS Converter |
|---|---|
| Description: | Preprocesses raw GPS trajectory data of public transit and transforms it to GTFS format. Provides a high-performance R port of the 'gps2gtfs' Python package by Aaivu (Ratneswaran et al., 2023) <doi:10.1109/ICCT56969.2023.10075789>, whose software description is published in Ratneswaran et al. (2025) <doi:10.1016/j.simpa.2025.100780>. Heavy computational tasks are offloaded to an extremely fast compiled Rust backend or a C++ (Rcpp) backend. Automatic backend selection prefers Rust, then Rcpp, and falls back to a slower pure R implementation utilizing 'data.table' and 'sf'. |
| Authors: | Egor Kotov [aut, cre, cph] (ORCID: <https://orcid.org/0000-0001-6690-5345>), Aaivu [cph] |
| Maintainer: | Egor Kotov <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.2.0 |
| Built: | 2026-07-27 12:54:40 UTC |
| Source: | https://github.com/e-kotov/gps2gtfs |
Cleans raw GPS or GTFS-Realtime vehicle position data: maps input columns to the canonical schema, removes records with zero coordinates or missing vehicle identifiers, deduplicates repeated pings, parses timestamps, and sorts by vehicle, date, and time.
g2g_clean_gps( raw_gps_df, projected = NULL, vehicle_col = "vehicle_id", time_col = "timestamp", tz = NULL, dedupe = TRUE, drop_missing_vehicle = TRUE )g2g_clean_gps( raw_gps_df, projected = NULL, vehicle_col = "vehicle_id", time_col = "timestamp", tz = NULL, dedupe = TRUE, drop_missing_vehicle = TRUE )
raw_gps_df |
A data.frame containing raw GPS data. Must include
|
projected |
Logical. Is the coordinates data already projected? Default is NULL (auto-detect). |
vehicle_col |
Character. Name of the column holding the vehicle
identifier. Default |
time_col |
Character. Name of the column holding the observation time.
Default |
tz |
Character. Timezone in which non- |
dedupe |
Logical. Drop repeated |
drop_missing_vehicle |
Logical. Drop rows with missing/empty vehicle
identifiers with a warning ( |
The canonical column names follow the GTFS-Realtime convention, so the
output of gtfsrealtime::read_gtfsrt_positions() is accepted directly.
Any other source (AVL exports, logger CSVs) can be mapped via
vehicle_col and time_col. Columns beyond the canonical set
(e.g. trip_id, route_id, stop_id) are passed through
untouched.
A sorted data.table with canonical columns id,
vehicle_id, latitude, longitude, timestamp,
speed, additional date and time_str columns, and all
other input columns passed through.
data(g2g_data_gps) cleaned_gps <- g2g_clean_gps(g2g_data_gps) head(cleaned_gps)data(g2g_data_gps) cleaned_gps <- g2g_clean_gps(g2g_data_gps) head(cleaned_gps)
A lightweight subset of raw bus GPS data from Kandy, Sri Lanka.
This dataset is intended for examples and testing of the gps2gtfs pipeline.
g2g_data_gpsg2g_data_gps
A data frame with 2167 rows and 6 variables:
Unique identifier for the GPS record.
Unique identifier for the tracking vehicle (bus). Column naming follows the GTFS-Realtime convention.
Timestamp of the GPS ping (UTC).
Latitude in WGS-84 degrees.
Longitude in WGS-84 degrees.
Recorded speed of the vehicle.
Original data from the Python gps2gtfs package.
Ratneswaran, S., & Thayasivam, U. (2023). Extracting potential Travel time information from raw GPS data and Evaluating the Performance of Public transit - a case study in Kandy, Sri Lanka. 2023 3rd International Conference on Intelligent Communication and Computational Techniques (ICCT), 1-7. doi:10.1109/ICCT56969.2023.10075789
Sample data containing the coordinates and metadata for bus stops.
This dataset works together with g2g_data_gps to extract stop times.
g2g_data_stopsg2g_data_stops
A data frame with 23 rows and 6 variables:
Unique identifier for the bus stop.
Route identifier the stop belongs to.
Direction of the route the stop serves.
Latitude in WGS-84 degrees.
Longitude in WGS-84 degrees.
Address or name of the stop location.
Original data from the Python gps2gtfs package.
Ratneswaran, S., & Thayasivam, U. (2023). Extracting potential Travel time information from raw GPS data and Evaluating the Performance of Public transit - a case study in Kandy, Sri Lanka. 2023 3rd International Conference on Intelligent Communication and Computational Techniques (ICCT), 1-7. doi:10.1109/ICCT56969.2023.10075789
Sample data containing the coordinates for bus route terminals. Used to define the start and end of trips.
g2g_data_terminalsg2g_data_terminals
A data frame with 2 rows and 4 variables:
Unique identifier for the bus terminal.
Name of the terminal.
Latitude in WGS-84 degrees.
Longitude in WGS-84 degrees.
Original data from the Python gps2gtfs package.
Ratneswaran, S., & Thayasivam, U. (2023). Extracting potential Travel time information from raw GPS data and Evaluating the Performance of Public transit - a case study in Kandy, Sri Lanka. 2023 3rd International Conference on Intelligent Communication and Computational Techniques (ICCT), 1-7. doi:10.1109/ICCT56969.2023.10075789
Retrieve the coverage diagnostics attached to the return value of
g2g_extract_trips or
g2g_extract_trips_and_stop_times. The diagnostics record how
many GPS pings, trip segments, and trips survived each stage of extraction,
so a non-random loss of coverage (a feed whose vehicle positions carry no
usable trip identity, routes the two-terminal model cannot segment, stops
out of buffer range) is visible as data instead of vanishing silently.
g2g_diagnostics(x)g2g_diagnostics(x)
x |
The value returned by
|
The extractors also emit a one-line warning() whenever a run loses
coverage worth surfacing, pointing back here — so unattended or agent-driven
pipelines notice the loss without having to inspect attributes. Duplicate
removal is treated as routine and never triggers that warning (it still
appears in the table). Suppress the warning with diagnostics_warn =
FALSE or options(gps2gtfs.diagnostics_warn = FALSE).
A g2g_diagnostics data.table with columns stage,
metric, and n (integer count; NA where a metric does
not apply to the run). It has a compact print method; treat it as a
normal data.table for programmatic use.
data(g2g_data_gps) data(g2g_data_terminals) data(g2g_data_stops) res <- g2g_extract_trips_and_stop_times( gps_data = g2g_data_gps, terminals_data = g2g_data_terminals, stops_data = g2g_data_stops, terminals_buffer_radius = 50, stops_buffer_radius = 30, stops_extended_buffer_radius = 50 ) g2g_diagnostics(res)data(g2g_data_gps) data(g2g_data_terminals) data(g2g_data_stops) res <- g2g_extract_trips_and_stop_times( gps_data = g2g_data_gps, terminals_data = g2g_data_terminals, stops_data = g2g_data_stops, terminals_buffer_radius = 50, stops_buffer_radius = 30, stops_extended_buffer_radius = 50 ) g2g_diagnostics(res)
Reads raw GPS and terminal CSV files, extracts trips, and optionally writes results to a CSV.
g2g_extract_trips( gps_data, terminals_data = NULL, terminals_buffer_radius = NULL, output_path = NULL, projected_crs = NULL, backend = "auto", projected = NULL, vehicle_col = "vehicle_id", time_col = "timestamp", tz = NULL, trip_col = NULL, direction_col = NULL, session_gap = 4 * 3600, segmentation = c("auto", "terminals", "layover"), layover_gap = 10 * 60, layover_radius = 50, diagnostics_warn = getOption("gps2gtfs.diagnostics_warn", TRUE) )g2g_extract_trips( gps_data, terminals_data = NULL, terminals_buffer_radius = NULL, output_path = NULL, projected_crs = NULL, backend = "auto", projected = NULL, vehicle_col = "vehicle_id", time_col = "timestamp", tz = NULL, trip_col = NULL, direction_col = NULL, session_gap = 4 * 3600, segmentation = c("auto", "terminals", "layover"), layover_gap = 10 * 60, layover_radius = 50, diagnostics_warn = getOption("gps2gtfs.diagnostics_warn", TRUE) )
gps_data |
A data.frame or path to the raw GPS CSV. |
terminals_data |
A data.frame or path to the terminal coordinates CSV.
Optional ( |
terminals_buffer_radius |
Numeric. Buffer radius for terminals (in meters). Only consumed by terminal-buffer segmentation; may be omitted otherwise. |
output_path |
Character. Optional path to write output trip features as CSV. Default is |
projected_crs |
Numeric. The EPSG code of a projected coordinate system
used for metric distance calculations. Required when
|
backend |
Character. The backend to use: |
projected |
Logical. Whether plain-table coordinates are already
projected. Out-of-bounds coordinates require explicit |
vehicle_col |
Character. Name of the vehicle identifier column in
|
time_col |
Character. Name of the timestamp column in |
tz |
Character. Timezone in which non- |
trip_col |
Character. Optional name(s) of column(s) holding supplied
trip identities (e.g. |
direction_col |
Character. Optional column giving each ping's travel
direction (e.g. GTFS-Realtime |
session_gap |
Numeric. Successive observations of a vehicle further
apart than this many seconds start a new driving session; trips never
span sessions. This replaces the former calendar-date boundary, so
overnight trips crossing midnight stay intact while overnight parking
still separates one day's operations from the next. Default 4 hours
( |
segmentation |
Character. How the raw-GPS spatial path cuts trips when
no |
layover_gap |
Numeric. Layover segmentation only: dwells longer than
this many seconds bound trips - whether a silent gap between successive
pings or a stationary spell (pings present but staying within
|
layover_radius |
Numeric. Layover segmentation only: a vehicle counts as stationary while successive pings stay within this many meters of the dwell's first ping (anchoring absorbs GPS jitter while parked). Default 50. |
diagnostics_warn |
Logical. Emit a one-line |
A data.table containing extracted trip features. start_time
and end_time are absolute POSIXct times in the timezone of
the input timestamps. Rows are returned in a stable, backend-invariant
order: sorted by the internal integer trip_id. The result carries
an attr(., "diagnostics") coverage table (see
g2g_diagnostics). See the "Inference tables, not GTFS files"
section of g2g_extract_trips_and_stop_times.
data(g2g_data_gps) data(g2g_data_terminals) trips <- g2g_extract_trips( gps_data = g2g_data_gps, terminals_data = g2g_data_terminals, terminals_buffer_radius = 50 )data(g2g_data_gps) data(g2g_data_terminals) trips <- g2g_extract_trips( gps_data = g2g_data_gps, terminals_data = g2g_data_terminals, terminals_buffer_radius = 50 )
Reads raw GPS, terminal, and stop CSV files, extracts trips and stop times, and optionally writes results.
g2g_extract_trips_and_stop_times( gps_data, terminals_data = NULL, stops_data, terminals_buffer_radius = NULL, stops_buffer_radius, stops_extended_buffer_radius, output_trips_path = NULL, output_stops_path = NULL, projected_crs = NULL, backend = "auto", projected = NULL, stop_direction_map = NULL, vehicle_col = "vehicle_id", time_col = "timestamp", tz = NULL, trip_col = NULL, direction_col = NULL, session_gap = 4 * 3600, segmentation = c("auto", "terminals", "layover"), layover_gap = 10 * 60, layover_radius = 50, return_trajectory = FALSE, diagnostics_warn = getOption("gps2gtfs.diagnostics_warn", TRUE) )g2g_extract_trips_and_stop_times( gps_data, terminals_data = NULL, stops_data, terminals_buffer_radius = NULL, stops_buffer_radius, stops_extended_buffer_radius, output_trips_path = NULL, output_stops_path = NULL, projected_crs = NULL, backend = "auto", projected = NULL, stop_direction_map = NULL, vehicle_col = "vehicle_id", time_col = "timestamp", tz = NULL, trip_col = NULL, direction_col = NULL, session_gap = 4 * 3600, segmentation = c("auto", "terminals", "layover"), layover_gap = 10 * 60, layover_radius = 50, return_trajectory = FALSE, diagnostics_warn = getOption("gps2gtfs.diagnostics_warn", TRUE) )
gps_data |
A data.frame or path to the raw GPS CSV. |
terminals_data |
A data.frame or path to the terminal coordinates CSV.
Optional ( |
stops_data |
A data.frame or path to the bus stops coordinates CSV.
Its |
terminals_buffer_radius |
Numeric. Buffer radius for terminals (in meters). Only consumed by terminal-buffer segmentation; may be omitted otherwise. |
stops_buffer_radius |
Numeric. Buffer radius for bus stops (in meters). |
stops_extended_buffer_radius |
Numeric. Extended buffer radius for bus stops (in meters). |
output_trips_path |
Character. Optional path to write output trip features as CSV. Default is |
output_stops_path |
Character. Optional path to write output stop times as CSV. Default is |
projected_crs |
Numeric. The EPSG code of a projected coordinate system
used for metric distance calculations. Required when
|
backend |
Character. The backend to use: |
projected |
Logical. Whether plain-table coordinates are already
projected. Out-of-bounds coordinates require explicit |
stop_direction_map |
Named character vector mapping each raw
stop-direction label to the terminal ID that trips in that direction
start from. Required whenever |
vehicle_col |
Character. Name of the vehicle identifier column in
|
time_col |
Character. Name of the timestamp column in |
tz |
Character. Timezone in which non- |
trip_col |
Character. Optional name(s) of column(s) holding supplied
trip identities (e.g. |
direction_col |
Character. Optional column giving each ping's travel
direction (e.g. GTFS-Realtime |
session_gap |
Numeric. Successive observations of a vehicle further
apart than this many seconds start a new driving session; trips never
span sessions. This replaces the former calendar-date boundary, so
overnight trips crossing midnight stay intact while overnight parking
still separates one day's operations from the next. Default 4 hours
( |
segmentation |
Character. How the raw-GPS spatial path cuts trips when
no |
layover_gap |
Numeric. Layover segmentation only: dwells longer than
this many seconds bound trips - whether a silent gap between successive
pings or a stationary spell (pings present but staying within
|
layover_radius |
Numeric. Layover segmentation only: a vehicle counts as stationary while successive pings stay within this many meters of the dwell's first ping (anchoring absorbs GPS jitter while parked). Default 50. |
return_trajectory |
Logical. Also return the ping-level trajectory
(cleaned GPS records with assigned |
diagnostics_warn |
Logical. Emit a one-line |
A list containing two data.tables: trips and
stop_times, plus trajectory when
return_trajectory = TRUE. All times (start_time,
end_time, arrival_time, departure_time) are absolute
POSIXct values in the timezone of the input timestamps — never
clock strings, so trips running past midnight stay unambiguous.
Row order is a stable, backend-invariant contract: trips are
ordered by the internal integer trip_id, and stop_times by
(trip_id, arrival_time, stop_id, departure_time) with every
remaining column appended as a final tie-breaker. The sort is therefore
total, so the three backends (Rust, Rcpp, pure R) return identical row
order for identical input and summaries built on the result are
reproducible regardless of backend or parallelism.
The result also carries an attr(., "diagnostics") coverage table
(see g2g_diagnostics).
The returned trips and stop_times use GTFS-style column
names but are *inference tables*, not valid trips.txt/
stop_times.txt: they carry no route_id, service_id,
stop_sequence, or GTFS clock strings ("HH:MM:SS", with
>24:00:00 for post-midnight stops). This is deliberate:
route_id, service_id, and canonical stop identities cannot be
inferred from coordinates alone, so gps2gtfs stops at inference
rather than synthesizing them.
Turning these tables into a standard-compliant feed — deriving
stop_sequence, attributing each trip to a service day, encoding
>24:00:00 clock strings, and linking or synthesizing IDs — is the
job of the companion package gtfsrt2static (a separate, optional
install), which yields a feed the MobilityData gtfs-validator
accepts:
res <- g2g_extract_trips_and_stop_times(...) events <- gtfsrt2static::snapshot_from_stop_times( res$stop_times, trip_id_col = "provided_trip_id" # keep official IDs ) feed <- gtfsrt2static::snapshot_assemble(events) # baseline: real IDs # or, with no planned feed to lean on: # feed <- gtfsrt2static::snapshot_scaffold(events, strict = TRUE) gtfsio::export_gtfs(feed, "realized_gtfs.zip") # gtfstools/tidytransit-ready
Both tables carry a provided_trip_id column: the caller's official
trip identity (the trip_col value, e.g. a GTFS-Realtime
trip_id) when the fast path is used, NA otherwise. It lets a
downstream assembler preserve official trip IDs instead of synthesizing
them. The internal integer trip_id remains the join key between the
two tables.
Both tables also carry an additive, versioned set of inference-label columns.
They are the contract slots for a future baseline-free
orientation/turnaround-detection stage and are currently always in the
“no detector applied” state (no such detector is enabled yet), so the
legacy direction column - retained unchanged - is what downstream
consumers read today:
orientation_idInteger 0/1, geometric travel
direction only; NA when unavailable/abstained (always NA
today).
orientation_statusFactor, never NA. "none"
(no orientation method applied) today; reserved values "ok" /
"single_group" / "abstain_<reason>" make a future
detector's abstentions observable rather than silent.
orientation_confidenceDouble in [0,1], NA
when orientation_id is NA (always NA today).
pattern_refCharacter K-way branch/short-turn variant
identity; reserved nullable (always NA today). The handoff to
gtfsrt2static's cross-trip stop-order stage.
start_anchor_ref, end_anchor_ref
Character discovered
turnaround-cluster ids; trips only (trip-level metadata,
not per-stop), NA today.
Downstream, gtfsrt2static maps orientation_id into the GTFS
direction_id (falling back to direction while
orientation_id is NA); pattern_ref rides through to C6;
anchors stay in C5. orientation_*/pattern_ref propagate from
trips onto stop_times on the internal trip_id, the same
mechanism as provided_trip_id.
data(g2g_data_gps) data(g2g_data_terminals) data(g2g_data_stops) result <- g2g_extract_trips_and_stop_times( gps_data = g2g_data_gps, terminals_data = g2g_data_terminals, stops_data = g2g_data_stops, terminals_buffer_radius = 50, stops_buffer_radius = 30, stops_extended_buffer_radius = 50, # Each stop-direction label maps to the terminal its trips start from. # BT01 is Kandy, BT02 is Digana. stop_direction_map = c( "Kandy-Digana" = "BT01", "Digana-Kandy" = "BT02" ) ) head(result$trips) head(result$stop_times)data(g2g_data_gps) data(g2g_data_terminals) data(g2g_data_stops) result <- g2g_extract_trips_and_stop_times( gps_data = g2g_data_gps, terminals_data = g2g_data_terminals, stops_data = g2g_data_stops, terminals_buffer_radius = 50, stops_buffer_radius = 30, stops_extended_buffer_radius = 50, # Each stop-direction label maps to the terminal its trips start from. # BT01 is Kandy, BT02 is Digana. stop_direction_map = c( "Kandy-Digana" = "BT01", "Digana-Kandy" = "BT02" ) ) head(result$trips) head(result$stop_times)
Converts the ping-level trajectory of extracted trips into a table shaped
like the GTFS shapes.txt file: one shape per trip, points in travel
order, with cumulative distance. Unlike planned shapes, these traces record
the geometry actually driven, including detours.
g2g_shapes_from_trips(trajectory, shape_id_prefix = "SHP_")g2g_shapes_from_trips(trajectory, shape_id_prefix = "SHP_")
trajectory |
A data.table of trajectory GPS records with columns
|
shape_id_prefix |
Character prefix for generated shape ids. Default
|
Obtain the trajectory by running
g2g_extract_trips_and_stop_times(..., return_trajectory = TRUE) and
using the $trajectory element of the result.
A data.table with GTFS-standard columns shape_id,
shape_pt_lat, shape_pt_lon, shape_pt_sequence,
shape_dist_traveled (meters).
Builds the stops_data input for
g2g_extract_trips_and_stop_times from a planned (baseline)
GTFS feed: all non-terminal stops served by a route, labeled with the
direction group they belong to. Direction labels are the terminal_id
a trip starts from, so they map onto the terminals derived by
g2g_terminals_from_gtfs without a manual
stop_direction_map. Note this means the labels are terminal
stop_ids, not GTFS direction_id values; they are not
interchangeable with a direction_col taken from the positions.
Trips that start at neither derived terminal have no direction group and are dropped with a warning, so stops served only by short turns or branch variants do not appear in the result.
g2g_stops_from_gtfs(gtfs, route_id)g2g_stops_from_gtfs(gtfs, route_id)
gtfs |
A GTFS feed object (named list of data.frames, as returned by
|
route_id |
A single route identifier present in |
A data.table with columns stop_id, latitude,
longitude, direction (the starting terminal_id of trips
serving the stop in that direction). Stops served in both directions
appear once per direction.
gtfs <- list( trips = data.frame(trip_id = c("t1", "t2"), route_id = "r1"), stop_times = data.frame( trip_id = c("t1", "t1", "t1", "t2", "t2", "t2"), stop_id = c("A", "S1", "B", "B", "S1", "A"), stop_sequence = c(1, 2, 3, 1, 2, 3) ), stops = data.frame( stop_id = c("A", "S1", "B"), stop_lat = c(7.29, 7.30, 7.31), stop_lon = c(80.63, 80.64, 80.65) ) ) g2g_stops_from_gtfs(gtfs, route_id = "r1")gtfs <- list( trips = data.frame(trip_id = c("t1", "t2"), route_id = "r1"), stop_times = data.frame( trip_id = c("t1", "t1", "t1", "t2", "t2", "t2"), stop_id = c("A", "S1", "B", "B", "S1", "A"), stop_sequence = c(1, 2, 3, 1, 2, 3) ), stops = data.frame( stop_id = c("A", "S1", "B"), stop_lat = c(7.29, 7.30, 7.31), stop_lon = c(80.63, 80.64, 80.65) ) ) g2g_stops_from_gtfs(gtfs, route_id = "r1")
Baseline-free helper: when no static GTFS feed exists, stop locations can
be estimated from GTFS-Realtime Vehicle Positions that carry a
stop_id annotation. Each stop's coordinates are the median position
of the pings observed at it — by default only pings labeled
STOPPED_AT, the most precise signal.
g2g_stops_from_positions(positions, statuses = "STOPPED_AT", min_obs = 3L)g2g_stops_from_positions(positions, statuses = "STOPPED_AT", min_obs = 3L)
positions |
A data.frame of vehicle positions (e.g. from
|
statuses |
Character vector of |
min_obs |
Integer. Minimum number of pings required per stop; stops
with fewer observations are dropped with a warning. Default |
The result covers the stop_id/latitude/longitude
columns of the stops_data input to
g2g_extract_trips_and_stop_times; the direction label
(starting terminal of trips serving the stop) must still be supplied, e.g.
from the RT trip_id/direction_id annotations or a baseline
feed. It also fills the spec-required stop_lat/stop_lon of a
scaffolded stops.txt.
A data.table with columns stop_id, latitude,
longitude, n_obs, sorted by stop_id.
Determines the two terminals of a route from a planned (baseline) GTFS
feed: the two most frequent trip endpoints (first/last stop per trip in
stop_times.txt). The result feeds directly into
terminals_data of g2g_extract_trips and
g2g_extract_trips_and_stop_times.
Frequency ranking cannot tell two ends of a line from two platforms of one
place, so the function warns when fewer than 90% of trip endpoints match
the two it derived (variants or branches) and when the two are close enough
together to be the same physical terminal under platform-level
stop_ids. Loop routes have no two terminals at all and error; use
segmentation = "layover" for those.
g2g_terminals_from_gtfs(gtfs, route_id)g2g_terminals_from_gtfs(gtfs, route_id)
gtfs |
A GTFS feed object (named list of data.frames, as returned by
|
route_id |
A single route identifier present in |
A data.table with columns terminal_id, latitude,
longitude.
gtfs <- list( trips = data.frame(trip_id = c("t1", "t2"), route_id = "r1"), stop_times = data.frame( trip_id = c("t1", "t1", "t2", "t2"), stop_id = c("A", "B", "B", "A"), stop_sequence = c(1, 2, 1, 2) ), stops = data.frame( stop_id = c("A", "B"), stop_lat = c(7.29, 7.31), stop_lon = c(80.63, 80.65) ) ) g2g_terminals_from_gtfs(gtfs, route_id = "r1")gtfs <- list( trips = data.frame(trip_id = c("t1", "t2"), route_id = "r1"), stop_times = data.frame( trip_id = c("t1", "t1", "t2", "t2"), stop_id = c("A", "B", "B", "A"), stop_sequence = c(1, 2, 1, 2) ), stops = data.frame( stop_id = c("A", "B"), stop_lat = c(7.29, 7.31), stop_lon = c(80.63, 80.65) ) ) g2g_terminals_from_gtfs(gtfs, route_id = "r1")