Support for Content:// Uri to access files on shared storage

[sachinkhatripro]

Use case

Content:// Uri is now the standard method for referencing files on android. But there is no direct way of processing Content:// Uri in Flutter/Dart. Currently, we can process File:// Uri and direct paths to a file but post Android R, it is practically useless.

This feature is a most urgent and must have requirement in Flutter.

Proposal

The current method of processing content based Uri is to go via a chain of Kotlin or java methods to **create a copy of the original file in app cache, which can be referenced with File:// Uri **. Or use a third party plugin(uri_to_file) on pub.dev which uses the same long winded kotlin route to do the same, creates a copy of the file in cache and lacks in some features. Not feasible for an app scanning and processing large files on local shared storage.

Third party plugins may or may not be maintained properly and may be lacking/buggy.

Flutter being a Google initiative, it is a given that a core Android functionality must be a part of Flutter too especially when the said functionality is being enforced in Android by Google itself. Otherwise, what's the point of using Flutter if we have to route through kotlin or java code for even basic features?

This feature is a most urgent and must have requirement in Flutter.

[devfemibadmus]

          https://github.com/flutter/flutter/issues/25659#issuecomment-1462496431

Closing, since File is part of Dart, not Flutter, so this would need to be filed at https://github.com/dart-lang/sdk/issues.

I suspect it would be closed there as well though, since it's not at all clear how many of the APIs in File would work. content >>URIs aren't files, they are their own distinct concept; that's why they have a different scheme in the first place.

Originally posted by @devfemibadmus in #143283 (comment)

I thought 🤔🤔🤔...

[sachinkhatripro]

          https://github.com/flutter/flutter/issues/25659#issuecomment-1462496431

Closing, since File is part of Dart, not Flutter, so this would need to be filed at https://github.com/dart-lang/sdk/issues.
I suspect it would be closed there as well though, since it's not at all clear how many of the APIs in File would work. content >>URIs aren't files, they are their own distinct concept; that's why they have a different scheme in the first place.
Originally posted by @devfemibadmus in #143283 (comment)

I thought 🤔🤔🤔...

It's better to let them come up with solution from scratch. They know better what they have under the hood and how things can be added.

[theskyblockman]

There is a package named uri_content which enables you to open an URI as a Stream<Uint8List> and get its size if possible. This system works through a local server between the native side and your Dart isolate(s) (like many things that aren't supported by Flutter...). The concept works well but it seems the package can only open content:// and http(s):// URIs for now. Also for some reason communicating locally between different ports of the same device on Android requires the INTERNET permission... This would also be needed to be mentioned when using something like that.

If we say this fixes the problem enough. Now would by any package requesting or providing Files in contexts where they doesn't fit would be one. Although this sounds "easier" to fix.

[stuartmorgan]

We have considered adding XFile support for content://; that's one potential option for exposing them in a performant way.

[stuartmorgan]

Also, a doc with relevant discussion: https://docs.google.com/document/d/12pJDtl0yubyc68UqKo2hQ7XJwv86_ixCjNemQ7ENCBQ/edit#heading=h.ebkwt1wlot63

[theskyblockman]

Also, a doc with relevant discussion: https://docs.google.com/document/d/12pJDtl0yubyc68UqKo2hQ7XJwv86_ixCjNemQ7ENCBQ/edit#heading=h.ebkwt1wlot63

Adding Android support to cross_file would certainly work, only how would cross_file get the data? Like I've said uri_content uses a local server to transfer the data, but I find this method to not be good enough for multiple reasons.
Platform channels would add a bit of overhead but is an option (marshaling isn't really ideal for something like raw file access).
If we want to be fancy, we could use jni which could be faster, when 1.0 will release (and start testing with it now).

[a-siva]

Not sure why this has a 'dependency:dart' label, based on the conversation in dart-lang/sdk#54878 the conclusion was to not treat these a File objects.

[theskyblockman]

I want to work on this as I find this feature important for my use case.

My question is who will have to implement the way to transfer the data from the native side to Dart? If we make cross_file able to take URIs as inputs (or anything) but then will cross_file transfer the data or the package (or app) who constructed the XFile? (i.e. file_selector)

This makes two options:

1. We would just use XFile.fromData or add a constructor for where a Sink<Uint8List> would be given instead to have better performance and let the packages implement it on their own.
2. If cross_file would transfer the data, first the package structure would need to change to add support for android packages and we could just use the unnamed constructor or create a constructor like XFile.fromContentUri (we would also need to add a static function to check if the content:// URIs are supported at runtime).

I personally prefer that we implement the second method as it would enable less duplicate code from other packages using XFile (file_picker now has support for XFIles for example).

If we opt for the second option I would like to have multiple ways to transfer the data available. My idea would be that we have the main package which wouldn't change and for example two other packages, one named cross_file_android_jni and another named cross_file_android_platform_channel. When cross_file would be added as a dependency, it would not have support for android, then one of those new packages would be added a dependency which will use an extension to add a constructor to XFile and implement their approach.
This would enable us to ship the platform channel approach now and when Dart JNI will release, release the JNI approach. If the JNI approach is better without any downsides we could phase out the dependency and keep only the JNI approach.

If what I am saying is possible, I would work first on the platform interface approach, then as I find the JNI approach way more efficient (if it is possible, I haven't found any implementation for Java Streams in JNI, so we would use polling or simply use platform channels to give update events to the Dart side), I would work to make it available for when Dart JNI will release (I haven't looked at the roadmap of the project so I don't know the ETA for its release, or if it will release at all).

[stuartmorgan]

My question is who will have to implement the way to transfer the data from the native side to Dart? If we make cross_file able to take URIs as inputs (or anything) but then will cross_file transfer the data or the package (or app) who constructed the XFile? (i.e. file_selector)

I would expect that cross_file would handle the data transfer, but it would need evaluation in a design document that investigated and compared options in more depth.

If we opt for the second option I would like to have multiple ways to transfer the data available. My idea would be that we have the main package which wouldn't change and for example two other packages, one named cross_file_android_jni and another named cross_file_android_platform_channel.

If cross_file moved to a federated plugin structure anyone could release whatever implementations they wanted to, but the Flutter team would not maintain two different implementations.

This would enable us to ship the platform channel approach now and when Dart JNI will release, release the JNI approach.

Having a single package and changing its internal implementation details later would also allow that, but without the overhead of managing two packages.

[theskyblockman]

Currently cross_file can be used without Flutter and as far as I know it is not possible to keep the package not federated and still keep Dart-only support does someone with more knowledge than me ensure that there is no other way to do this?

Here is the first design document for the feature (I am not entirely sure of where ContentUri could be depending on if we make cross_file federated or not): https://docs.flutter.dev/go/cross_file-content-uri-implementation

I have also made a draft of what ContentUri could be in this commit for option 1.

[theskyblockman]

I have evaluated both options from my document and found the second more appealing:
It would keep cross_file as an abstraction layer and would not break backwards-compatibility as the edits directly to cross_file are minimal

I have worked on option 2 and looks way better (saveTo causes me some trouble on web because of my lack of knowledge on this platform in particular with blobs).