File download issue with files size 2GB+

[deepktp]

Versions Information:

react-native: 0.77.1

react: 18.3.1

react-native-blob-util: 0.19.11 | 0.21.2 (Tried both versions)

device OS: Android 12 | 13 | 14

device type: Physical

Issue:

The file download promise never resolve in case of trying to downloading files larger then 2GB+ (Approx).

Used Code:

const task = RNBU.config({
        fileCache: true,
        appendExt: 'bin',
        path:  storagePath,
    }).fetch('GET', fileLink); //https://huggingface.co/bartowski/Phi-3.5-mini-instruct-GGUF/resolve/main/Phi-3.5-mini-instruct-Q4_K_M.gguf  (file size 2.39GB)

    task.progress((received, total) => {
        console.log(received, total);
        progress(parseInt(received, 10) / parseInt(total, 10));
    });

    task.then(res) => {
         console.log(res);
    }

###What's happing

1. I am downloading files larger then 2 gb in progress callback the total is fixed to 2147483647 (Approx 1.9999GB) (Also The 32-bit Java int can
   go to a maximum of 2,147,483,647), but received works perfectly fine (it's values go beyond total with file size 2GB+).

2. while downloading large file received(process callback params) works correctly even for 2GB+ file

3. Process callback keep triggering until full received is equal to real file size and then it is not called

4. but still task.then (Promise resolved) never triggered

5. !!!!Important: Full file is downloaded and saved even though promise never resolve

[deepktp]

File Response handler

i have made some modifications and now it works fine

issue was to fix some sort of OKio bug content length's max value was set to Integer.MAX_VALUE

!!!! Caution : I don't have any particular knowledge of Java and don't know any breaking effects !!!

package com.ReactNativeBlobUtil.Response;

import androidx.annotation.NonNull;

import com.ReactNativeBlobUtil.ReactNativeBlobUtilConst;
import com.ReactNativeBlobUtil.ReactNativeBlobUtilProgressConfig;
import com.ReactNativeBlobUtil.ReactNativeBlobUtilReq;
import com.facebook.react.bridge.Arguments;
import com.facebook.react.bridge.ReactApplicationContext;
import com.facebook.react.bridge.WritableMap;
import com.facebook.react.modules.core.DeviceEventManagerModule;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;

import okhttp3.MediaType;
import okhttp3.ResponseBody;
import okio.Buffer;
import okio.BufferedSource;
import okio.Okio;
import okio.Source;
import okio.Timeout;

/**
 * Created by wkh237 on 2016/7/11.
 */
public class ReactNativeBlobUtilFileResp extends ResponseBody {

    String mTaskId;
    ResponseBody originalBody;
    String mPath;
    long bytesDownloaded = 0;
    ReactApplicationContext rctContext;
    FileOutputStream ofStream;
    boolean isEndMarkerReceived;

    public ReactNativeBlobUtilFileResp(ResponseBody body) {
        super();
        this.originalBody = body;
    }

    public ReactNativeBlobUtilFileResp(ReactApplicationContext ctx, String taskId, ResponseBody body, String path, boolean overwrite) throws IOException {
        super();
        this.rctContext = ctx;
        this.mTaskId = taskId;
        this.originalBody = body;
        assert path != null;
        this.mPath = path;
        this.isEndMarkerReceived = false;
        if (path != null) {
            boolean appendToExistingFile = !overwrite;
            path = path.replace("?append=true", "");
            mPath = path;
            File f = new File(path);

            File parent = f.getParentFile();
            if (parent != null && !parent.exists() && !parent.mkdirs()) {
                throw new IllegalStateException("Couldn't create dir: " + parent);
            }

            if (!f.exists())
                f.createNewFile();
            ofStream = new FileOutputStream(new File(path), appendToExistingFile);
        }
    }

    @Override
    public MediaType contentType() {
        return originalBody.contentType();
    }

    @Override
    public long contentLength() {
        if (originalBody.contentLength() > Integer.MAX_VALUE) {
            // This is a workaround for a bug Okio buffer where it can't handle larger than int.
            return Integer.MAX_VALUE;
        }
        return originalBody.contentLength();
    }

    public boolean isDownloadComplete() {
        return (bytesDownloaded == originalBody.contentLength()) // Case of non-chunked downloads
                || (contentLength() == -1 && isEndMarkerReceived); // Case of chunked downloads
    }

    @Override
    public BufferedSource source() {
        ProgressReportingSource countable = new ProgressReportingSource();
        return Okio.buffer(countable);
    }

    private class ProgressReportingSource implements Source {
        @Override
        public long read(@NonNull Buffer sink, long byteCount) throws IOException {
            try {
                byte[] bytes = new byte[(int) byteCount];
                long read = originalBody.byteStream().read(bytes, 0, (int) byteCount);
                bytesDownloaded += read > 0 ? read : 0;
                if (read > 0) {
                    ofStream.write(bytes, 0, (int) read);
                } else if (contentLength() == -1 && read == -1) {
                    // End marker has been received for chunked download
                    isEndMarkerReceived = true;
                }
                ReactNativeBlobUtilProgressConfig reportConfig = ReactNativeBlobUtilReq.getReportProgress(mTaskId);

                if (contentLength() != 0) {

                    // For non-chunked download, progress is received / total
                    // For chunked download, progress can be either 0 (started) or 1 (ended)
                    float progress = (contentLength() != -1) ? bytesDownloaded / contentLength() : ((isEndMarkerReceived) ? 1 : 0);

                    if (reportConfig != null && reportConfig.shouldReport(progress /* progress */)) {
                        if (contentLength() != -1) {
                            // For non-chunked downloads
                            reportProgress(mTaskId, bytesDownloaded, originalBody.contentLength());
                        } else {
                            // For chunked downloads
                            if (!isEndMarkerReceived) {
                                reportProgress(mTaskId, 0, originalBody.contentLength());
                            } else {
                                reportProgress(mTaskId, bytesDownloaded, bytesDownloaded);
                            }
                        }
                    }

                }

                return read;
            } catch (Exception ex) {
                return -1;
            }
        }

        private void reportProgress(String taskId, long bytesDownloaded, long contentLength) {
            WritableMap args = Arguments.createMap();
            args.putString("taskId", taskId);
            args.putString("written", String.valueOf(bytesDownloaded));
            args.putString("total", String.valueOf(contentLength));
            rctContext.getJSModule(DeviceEventManagerModule.RCTDeviceEventEmitter.class)
                    .emit(ReactNativeBlobUtilConst.EVENT_PROGRESS, args);
        }

        @Override
        public Timeout timeout() {
            return null;
        }

        @Override
        public void close() throws IOException {
            ofStream.close();

        }
    }

}

i have modified reportProgress parameter (Only contentLength() with originalBody.contentLength()) and also for isDownloadComplete methods first condition

now react-native will receive correct total value and isDownloadComplete will be triggred at final complete

[RonRadtke]

Could you create a pull-request? Makes it easier to check and test