NAME
    Scalar::ValueTags::LineageTracking - track data lineages using ValueTags

SYNOPIS
        # setup: choose type of data to store in tags
        my $lt = Scalar::ValueTags::LineageTracking->new( tag_type => 'string' );
        my $lt = Scalar::ValueTags::LineageTracking->new( tag_type => 'ref' );

        # default tag_type: string
        my $lt = Scalar::ValueTags::LineageTracking->new();

        ### append tracking data sources when data is received from external sources
        my $names = $dbh->selectcol_arrayref('select name from user where user_id = %s", undef, $id);
        my $name = $names->[0];

        # using 'ref' tag_type
        $lt->add_data_sources(\$name,
            [
                { table => 'user', column => 'name', id => $id, file => __FILE__, line => __LINE__ },
                ...,
            ]
        );

        # using 'string' tag_type
        my $json = JSON->new->utf8->canonical->pretty(0);
        $lt->add_data_sources( \$name,
            [
                $json->encode( { table => 'user', column => 'name', id => $id, file => __FILE__, line => __LINE__ } ),
                ...,
            ],
        );

        ### clear previous data sources and set data source
        # 'ref' tag_type
        $lt->set_data_sources( \$name,
            [
                { table => 'user', column => 'name' },
                ...,
            ]
        );
        # 'string' tag_type
        $lt->set_data_sources( \$name,
            [
                $json->encode( { table => 'user', column => 'name' } ),
                ...,
            ]
        );

        ### data sources are propagated any time data is used
        $lt->add_data_source(\$name, 'tag-one');
        $lt->add_data_source(\$honorif, 'tag-two');
        my $salutation = "Hello, $honorif $name";

        # returns data sources set on both $honorif and $name:
        my $sources = $lt->get_data_sources(\$salutation);

        ### retrieve and clear data sources, for reporting
        my $sources = $lt->get_and_clear_data_sources(\$salutation);
        send_lineage( { sources => $sources, var => 'salutation', file => __FILE__, line => __LINE__ } );

DESCRIPTION
    This module uses "Scalar::ValueTags" (propagated value magic) to
    implement data flow tracking that can be used to populate data lineage
    systems such as OpenLineage.

    "LineageTracking" allows arbitrary tracking data to be attached to data
    values when the values are received from an external system, propagates
    the tracking data any time that other data is derived from the original
    data, and captures the tracking data from the value when it is sent to
    an external sink.

    The tracking data may be either an unblessed Perl reference or a
    serialized data string, depending on the "tag_type" parameter.

    When the tracking items are propagated, they are de-duplicated using
    either the string or the refaddr of the data structure that was set in
    "set_data_sources".

    The tracking data may be formatted in any way, such as OpenLineage.

  Implementation
    For the "hash" "tag_type", the "SVTAGS_UNIQUE_HASH" behavior is used,
    allowing string-serialized data to be used as tags. The tags are
    de-duplicated by the string value, so the serialization must generate
    canonical strings.

    For the "ref" "tag_type", the "SVTAGS_UNIQUE_REF_ARRAY" behavior is
    used, allowing any arbitray Perl reference to be used as tags. The tags
    are de-duplicated by the "refaddr" of the reference rather than the
    contents, so identical tags must use the same Perl reference.

PARAMETERS
  tag_type
    The "tag_type" determines the type of the tags passed to
    Scalar::ValueTags.

    *   "string"

        Each value tag must be a string suitable for use as a hash key.
        De-duplication is done automatically since the string is used as a
        hash key.

    *   "ref"

        Each value tag must be a Perl reference. De-duplication is done by
        the "refaddr" of the reference.

METHODS
  add_data_sources
        # using 'string' tag type
        $lt->add_data_sources( \$var, [ $string1, $string2, ... ] );

        # using 'ref' tag type
        $lt->add_data_sources( \$var, [ { this => 1 }, { that => 2 } ] );

    Append the given data sources to the value of the given $var. Each data
    source must match the tag_type set upon instantiation.

    Does not return anything.

  clear_data_sources
        my $data_sources = $lt->clear_data_sources(\$var);
        for my $data_source (@$data_sources) { ... }

    Clears all data sources on the value in given $var.

    Returns an arrayref of all of the data sources that previously existed
    on the value in $var.

  get_data_sources
        my $data_sources = $lt->get_data_sources(\$var);
        for my $data_source (@$data_sources) { ... }

    Returns an arrayref of all data sources set for the value in the given
    $var. Includes all propagated tags from all data values that were used
    to calculate the value of $var.

  set_data_sources
        # using 'string' tag type
        my $prev_sources = $lt->set_data_sources( \$var, [ $string1, $string2, ... ] );

        # using 'ref' tag type
        my $prev_sources = $lt->set_data_sources( \$var, [ { this => 1 }, { that => 2 } ] );

    Set the data sources for the value of the given $var to be the given
    data sources, clearing any pervious data sources. Each data source must
    match the tag_type set upon instantiation.

    Returns an arrayref of the previous data sources that were set on the
    value in $var.

LICENSE
    This module is released under the same terms as Perl itself.

AUTHORS
    *   Noel Maddy <zhtwnpanta@gmail.com>