Information on compiling addons on IBM PPC system, under Conda

[paulobaptista]

I wanted to shared the steps to compile addons with a Conda environment that loads TF, CUDA, CUDNN

Cuda Driver and Hardware
Driver Version: 440.33.01
CUDA Version: 10.2
GPU: Tesla V100
Conda Environment
tensorflow 2.1.3
tensorflow-gpu 2.1.3
cudatoolkit 10.2.89
cudatoolkit-dev 10.2.89
cudnn 7.6.5_10.2
bazel 0.29.1

Process
to meet addons requirements based on versions of installed CUDA, Tensorflow versiosn, we need to install 0.9.1 version of addons
activate conda environment with proper versions of tensorflow, cuda, cudnn
git clone https://github.com/tensorflow/addons.git
cd addons
git checkout tags/v0.9.1
set environment variables
export TF_NEED_CUDA="1"
export TF_CUDA_PATHS=$CONDA_PREFIX/cuda <---(this was the critical step)
export TF_CUDA_VERSION="10"
export TF_CUDNN_VERSION="7"
export CUDA_TOOLKIT_PATH=$CONDA_PREFIX
export CUDNN_INSTALL_PATH=$CONDA_PREFIX
python3 ./configure.py

The following applies for v0.9.1 of the build...

edit build_deps/toolchains/gpu/crosstool/BUILD.tpl
add the "ppc" line to the following block
cc_toolchain_suite(
name = "toolchain",
toolchains = {
"local|compiler": ":cc-compiler-local",
"ppc": ":cc-compiler-local",
"k8": ":cc-compiler-local",
},
)

run build and install

bazel build --enable_runfiles build_pip_pkg
bazel-bin/build_pip_pkg artifacts
pip install artifacts/tensorflow_addons-*.whl

[bhack]

Thanks, but we don't officially support conda here and in TF. @WindQAQ @seanpmorgan Can we collect this info somewhere?

[seanpmorgan]

There was an early hope to package on conda forge:
#201

The rest of the tf ecosystem hasn't done it either (except TF core which is managed by Anaconda). I'm interested but concerned for the amount of work

[bhack]

Esri has an Addons conda package. Is It still maintained?
https://anaconda.org/esri/tensorflow-addons

[bhack]

Some other conda packages https://anaconda.org/search?q=Addons+tensorflow

[seanpmorgan]

I didn't know of those packaging channels that support Addons. Given our availability I think it's best we let the community handle Conda builds (same thing TF core does, but Anaconda does it). Closing this as we have no way to test CI for a power PC build.

Thanks for posting this though I'm sure it'll be helpful for others. I wouldn't be opposed to a PR which adds this either since it can't really hurt the existing builds. Closing for now but happy to review PR if there's community interest.