tensorflow-io 0.10.0 release

[yongtang]

Since TensorFlow 1.15.0 and 2.0.0 has been released, we are in the process of releasing 0.8.0 (tracking #459) and 0.9.0 (tracking #517).

As we use the same code base for 0.8.0 (for TF 1.15.0) and 0.9.0 (for TF 2.0.0), we took quite a few compromises to maintain compatibility.

I think it makes sense for use to release a 0.10.0 soon, to create a clean baseline moving forward with TF 2.0. Several things I can think of for 0.10.0:

1. Move all tests in eager mode (and rename all tests files by removing _eager.py suffix).
   Note: Even though we only support eager mode, the graph mode is automatically tested in tf.keras (tf.keras use graph mode internally).
2. Remove all instances of tf.compat.v1. (as we only support 2.0 moving forward.
3. Remove all deprecated APIs.

Finally, I want to bring up the API discussion in #537. I think a namespace cleanup and consolidation is overdue for tensorflow-io?

@dmitrievanthony @terrytangyuan @BryanCutler @vlasenkoalexey Any comments or suggestions?

[BryanCutler]

Even though we only support eager mode, the graph mode is automatically tested in tf.keras (tf.keras use graph mode internally)

So should we make sure that each tests runs tf.keras also?

The steps for moving forward sound good to me and the API cleanup seems reasonable.

[yongtang]

Thanks @BryanCutler. I think it makes sense to make sure that each dataset runs with tf.keras. We could simplify as we only need to check if the tf.data + tf.keras pipeline is working.

I think something like a simple one or two layer of tf.keras, and a tf.data.Dataset of 10 elements should be good enough. It might even make sense to keep a template of tests so that it applies to every Dataset we implement. We could use tf.data.Dataset.take(10) to control the number of element, and use dataset.map(tf.cast()) to cast to float32.

[yongtang]

Some other thought during the trip to Contributor Summit:

API stability

Eventually we have to guarantee API is stable after 1.0, I think we have a long way to go (I could see we will reach 0.20 or later before 1.0 is released):

• We could designate v0, v1, v2 for stable releases, and experimental for non-stable release in python space.
• We may need to get to C++ API stability. As we discussed I think primitive kernel ops is a lot easier to maintain. That is also the reason I am playing with basic tf.data.Dataset + map function.

Other Language Bindings and Graph Mode

Eventually other language bindings will come in play. We could work with other SIGs so other language bindings does not need to be in tensorflow/io repo. However, in order to support other language bindings we will need to support graph mode saving, so that at least the ops could be saved as a graph and played out with other languages.

• This is also one of the reasons I am promoting basic kernel ops as it is a lot easier to use it for other language bindings.

Namepace

Our python namespace is bloated as we didn't envision we could have a lot of implementation span across so many areas and communities.

• We want to address the issue but some of the namespace have already been grandfathered in. To clean up we will need to deprecate some namespaces.

C++ template hard to use

We all know it is hard to implement in C++ and this to an extent is a big reason we don't have lots of C++ contributions so far.

By the way, currently our ratio of lines of code for C++ vs Python is about 51% vs 44% (based on GitHub metric). Would really love to see our repo eventually becomes Python > C++.

• With basic kernel ops the situation is a lot better. Though some C++ code is always needed.

• During the trip in Contributor Summit, there was some discussions about C++ template. I introduced quite a few C++ template in tensorflow/io, hoping to reduce the heavy C++ burden.

• But it looks like this might be causing more confusion. I will play around and see if we could avoid C++ template and still make C++ contributions easier.

/cc @terrytangyuan @BryanCutler

[yongtang]

The TF 2.1 release candidate was planned for 11/07:
https://groups.google.com/a/tensorflow.org/forum/#!topic/developers/DK4QK25ViA4

It might have been delayed though it looks like tensorflow's r2.1 branch has been created yesterday. For that I believe TF 2.1 rc0 (or rc1) will be released next week. That means TF 2.1 final might be released in another several weeks (likely December).

Relate to tensorflow-io, I am thinking it makes sense for us to release a 0.10.0 before the final release of TF 2.1 (as we have added many new features since 0.9.0).

We could then release 0.11.0 at the same time TF 2.1 is released.

/cc @BryanCutler @terrytangyuan @vlasenkoalexey @dmitrievanthony

[yongtang]

With TF 2.1.0rc0 out, we could prepare for a 0.10.0 release (match TF 2.0).

After TF 2.1.0 final we then release 0.11.0 for TF 2.1.

[yongtang]

Created PR #661 for the RELEASE note update.

[yongtang]

With #661 merged in, we are pretty much ready to have the next release. The candidate binaries are stored in:
https://www.dropbox.com/home/Apps/tensorflow-io-nightly/release/2265/wheelhouse

I will do some additional local testing. If everything is fine and there is no objections, we will just push for a release today.

/cc @terrytangyuan @BryanCutler @vlasenkoalexey @dmitrievanthony

[yongtang]

All looks good, will cut the release. Here is the binary:

bf54eec19b05db5d4b45983c0048366474df7b3e6d86a9309d3e479b54ee4713  \
    tensorflow_io-0.10.0-cp27-cp27m-macosx_10_13_x86_64.whl
501978815f8faacd1d98eacc918815fccc79f82e2ea883be7f3b471e2060a9af  \
    tensorflow_io-0.10.0-cp27-cp27mu-manylinux2010_x86_64.whl
1752d312d42f79a14ba3321b2aaf727bf9c9db6f7fbdcaab775a2f2f2fa8768d  \
    tensorflow_io-0.10.0-cp35-cp35m-macosx_10_13_x86_64.whl
86bc749950650367459c120f193b3c347da9abee9de2c2ecafb0499b8408f2d4  \
    tensorflow_io-0.10.0-cp35-cp35m-manylinux2010_x86_64.whl
e523ad7c8e2c0c9c9380ee022b2484ce663a6dd82ac49884f667000354d55fe2  \
    tensorflow_io-0.10.0-cp36-cp36m-macosx_10_13_x86_64.whl
807641a55fc9d797df4733b8d95f07b4fb0aa2f598306b3507d2be98e1fa09a0  \
    tensorflow_io-0.10.0-cp36-cp36m-manylinux2010_x86_64.whl
bf45fedb73ee967a758199d544e81e1e36e3f6ab207465c961857ef7f10a43fe  \
    tensorflow_io-0.10.0-cp37-cp37m-macosx_10_13_x86_64.whl
2879e2b0af7ae6a65f4c916b999455a55527010867bfa23bc43bccd31c1ce3e4  \
    tensorflow_io-0.10.0-cp37-cp37m-manylinux2010_x86_64.whl

[yongtang]

tensorflow-io 0.10.0 has been released:

https://github.com/tensorflow/io/releases/tag/v0.10.0

Thanks all! 👍 🎉