Skip to content
This repository was archived by the owner on Jul 31, 2023. It is now read-only.

Generate TFRecords only if data exists in a split. #25

Merged
merged 1 commit into from
Sep 18, 2020
Merged

Generate TFRecords only if data exists in a split. #25

merged 1 commit into from
Sep 18, 2020

Conversation

cfezequiel
Copy link
Contributor

Resolves #16

@cfezequiel cfezequiel requested a review from mbernico September 18, 2020 16:49
@cfezequiel cfezequiel mentioned this pull request Sep 18, 2020
Closed
mbernico
mbernico reviewed Sep 18, 2020
View reviewed changes
tfrecorder/beam_pipeline.py
return df[constants.SPLIT_KEY].value_counts().to_dict()


def _transform_and_write_tfr(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like, this, it cleans things up quite a bit.

mbernico
mbernico approved these changes Sep 18, 2020
View reviewed changes
Copy link
Contributor

@mbernico mbernico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mbernico mbernico merged commit 7779bee into master Sep 18, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@mbernico mbernico mbernico approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Train, Test and Val TFRecords files partitions always created
2 participants
@cfezequiel @mbernico