Fetch partial

[VolkerHartmann]

Unfortunately all files have to be fetched to validate bag due to specification of kunze!

Only the last (two?) commits please.

[cneud]

Closing as this was included with #94.

[cneud]

Need to adapt according to #106 (comment)

[kba]

Best to wait for @VolkerHartmann, it's not a high priority issue, concerns only the validators AIFAICS. In the end, it cannot be an error if that rule is broken because real data is messy.

[cneud]

Exactly. But would be good if validation does catch this. IME most real data is messy ;)

[EEngl52]

@cneud @kba do you still want to adapt this with @VolkerHartmann no longer available?

[cneud]

@EEngl52 This has implications on spec (bagit), so we should not close without a solution (even if not currently worked on).

[kba]

I don't see the use case for the fetch.txt mechanism or a strict "if in METS then in bag/fetch.txt" rule. We're using relative files anyway virtually exclusively and OCRD-ZIP is meant to store the OCR processing results, not be a 1:1 mapping of the METS. The main drawback of not using fetch.txt is that the bags are very large. The main advantage is that the bags are fully self-contained, no need for (possibly failing) network requests after receiving the bag.

I think there are other areas that are under- or unspecified that should be a much higher priority, such as the question how we can map back the relative filenames to HTTP URL for DFG Viewer compatibility or how we integrate a future workflow engine/parallelization setup with provenance etc.

[cneud]

100% ACK, but before closing here we should probably still mention this in the relating docs and what about the validation?

[kba]

You're right, in fact I think we should set `Allow-Fetch.txt: false in the bagit profile and remove any mention of it and partial OCRD-ZIP. BTW the proposed change made it into the spec anyway and has been for two years.

I think the validation is actually easier without fetch.txt because WYSIWYG, no network problems interfering.

[bertsky]

Is this related to OCR-D/core#323 by any chance?

[kba]

Is this related to OCR-D/core#323 by any chance?

Yes and by extension to OCR-D/core#176

[cneud]

Since afaict OCR-D/core#323 is unaffected, this should also be closed with #182.

[kba]

Superseded by #182