Skip to content
This repository was archived by the owner on Jul 31, 2023. It is now read-only.

Improve Installation Process [setup.py file location] #21

Closed
mchrestkha opened this issue Sep 11, 2020 · 5 comments · Fixed by #23
Closed

Improve Installation Process [setup.py file location] #21

mchrestkha opened this issue Sep 11, 2020 · 5 comments · Fixed by #23
Assignees
Labels
bug Something isn't working enhancement New feature or request

Comments

@mchrestkha
Copy link

Is your feature request related to a problem? Please describe.
The most likely installation process for most users may result in the Dataflow runner not working as the setup.py file for the Dataflow workers will not be in the right place.

Describe the solution you'd like
Assume users will only pip install and not clone the repo. Place the setup.py file into the main library folder and update setup.py location in beam_pipeline.py

Describe alternatives you've considered
Got it working after moving the setup.py file into the main python ..python3.7/sitepackages/ folder.

Additional context
Add any other context or screenshots about the feature request here.

@mchrestkha mchrestkha added the enhancement New feature or request label Sep 11, 2020
@mbernico mbernico self-assigned this Sep 11, 2020
@mbernico
Copy link
Contributor

Thanks for the report, recreating this now.

@mbernico mbernico added the bug Something isn't working label Sep 11, 2020
@mbernico
Copy link
Contributor

Successfully reproduced, digging into the issue.

@mbernico
Copy link
Contributor

Issue is at beam_pipeline.py 98.

if runner == 'DataflowRunner':
options_dict['setup_file'] = _get_setup_py_filepath()

Setup file isn't installed when TFRecorder is installed via pypi / whl (works with git)

@mbernico
Copy link
Contributor

mbernico commented Sep 14, 2020

It looks like TFDV solves this problem by prompting the user to download the whl and supply it as a command line option, passing it to beam pipeline options as an extra package. This is 'less easy' than I'd like it to be but some variation of this is the best idea I have for a fix at this time.

More info: https://www.tensorflow.org/tfx/data_validation/get_started

mbernico added a commit that referenced this issue Sep 14, 2020
This was referenced Sep 14, 2020
@mbernico
Copy link
Contributor

This should now be resolves in git @mchrestkha

I'll push it to pypi in just a few minutes as well.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants