Done
Pinned fields
Click on the next to a field label to start pinning.
Details
Details
Assignee
Jeremy McCormick
Jeremy McCormickReporter
Gregory Dubois-Felsmann
Gregory Dubois-FelsmannStory Points
2
RubinTeam
Data Engineering
Components
Priority
Checklist
Checklist
Created November 28, 2023 at 6:10 PM
Updated February 5, 2024 at 1:54 AM
Resolved January 23, 2024 at 1:13 AM
Felis YAML files with more than one foreign-key declaration against a single table do not seem to be processed correctly.
The above was an early misdiagnosis of the problems with foreign-key links. It turns out there were two independent problems in the Felis tooling, both related to forward references:
The internal data structure from JSON-LD processing ended up in a surprising state when forward references (from foreign-key relations) were added: the full definition of a column was inserted in the structure at the point of first reference, rather than in the definition of the table to which the column belonged. This problem was fixed by the PR linked to this ticket: https://github.com/lsst/felis/pull/26
After that was fixed, we learned that the processing of the data structure into TAP_SCHEMA data was also affected by the forward reference. This was fixed under DM-42293 by changing the TAP processing to be done in two phases: all tables and columns processed first, and only then processing foreign-key relations.