Cygwin internal installation should add winsymlinks:native to CYGWIN

[dra27]

This is a first step towards improved symlink support. We should ultimately be advising that developers enable Developer Mode (which allows symlinks to be used without elevation), and we should also have a better fallback for tarballs which include symlinks, but in the meantime, using the environment setup from #5781:

C:\>opam install ocamlbuild
The following actions will be performed:
=== install 1 package
  * ocamlbuild 0.14.2

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>

#=== ERROR while fetching sources for ocamlbuild.0.14.2 =======================#
OpamSolution.Fetch_fail("C:\\Devel\\opam-3\\opam.exe: \"lstat\" failed on C:\\Users\\DRA\\AppData\\Local\\Temp\\opam-126020-bae9bd\\ocamlbuild-0.14.2\\examples\\07-dependent-projects\\libdemo: No such file or directory")


<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
┌─ The following actions failed
│ ▼ fetch ocamlbuild 0.14.2
└─
╶─ No changes have been performed

The issue is that Cygwin will have used its own symlink format (most likely a WSL symlink) which native Windows opam can't read. The workaround is easy:

C:\>set CYGWIN=winsymlinks:native

C:\>opam install ocamlbuild
The following actions will be performed:
=== install 1 package
  * ocamlbuild 0.14.2

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
▼ retrieved ocamlbuild.0.14.2  (cached)
* installed ocamlbuild.0.14.2
Done.

At the same point as we add the internal Cygwin bin to the PATH, we should ensure that winsymlinks:native (or winsymlinks:nativestrict, if the user has elected to set that) are set and remove either winsymlinks:sys or winsymlinks:lnk if found (i.e. it's part of scrubbing the environment.). Cygwin's CYGWIN variable is documented here and its symlink support is documented here.

[jacj-simcorp]

I am getting an error during opam install ocp-indent (opam is 2.2 beta2):

OpamSolution.Fetch_fail("\027[33m#\027[0m \027[33mpath\027[0m        C:\\dev\n\027[33m#\027[0m \027[33mcommand\027[0m     C:\\cygwin-ml\\bin\\tar.exe xfz /cygdrive/c/Users/JACJ/AppData/Local/opam/download-cache/md5/70/70db6649a8c08a682ad63730c9752e31 -C /cygdrive/c/Users/JACJ/AppData/Local/Temp/opam-25096-388ea4\n\027[33m#\027[0m \027[33mexit-code\027[0m   2\n\027[33m#\027[0m \027[33menv-file\027[0m    C:\\Users\\JACJ\\AppData\\Local\\opam\\log\\log-25096-e1d27d.env\n\027[33m#\027[0m \027[33moutput-file\027[0m C:\\Users\\JACJ\\AppData\\Local\\opam\\log\\log-25096-e1d27d.out\n\027[33m### output ###\n\027[0m\027[33m# \027[0m/usr/bin/tar: ocp-indent-1.8.1/tests/inplace/link.ml: Cannot create symlink to 'otherfile.ml': No such file or directory\n\027[33m# \027[0m/usr/bin/tar: ocp-indent-1.8.1/tests/inplace/link2.ml: Cannot create symlink to 'link.ml': No such file or directory\n\027[33m# \027[0m/usr/bin/tar: Exiting with failure status due to previous errors\n"
$ echo $CYGWIN
winsymlinks:nativestrict

Note that native is no better.

Linking works (I have developer mode enabled):

touch test
cmd /c mklink test2 test
symbolic link created for test2 <<===>> test

It worked fine for ocamlbuild, mind you.

Should I create a new issue or is there something I missed?

[kit-ty-kate]

ocp-indent-1.8.1/tests/inplace/link.ml: Cannot create symlink to 'otherfile.ml': No such file or directory

the reason ocp-indent fails and not ocamlbuild here is that, from what i understand, on Windows it is not possible to create a symlink to a non-existant file and tar extracts file in some pre-defined order.

In ocamlbuild, tar extracted the target before the symlink so it works fine.
In ocp-index, the link is placed before the target in lexicographic order so it fails.

In this particular case I would say this is an error in the package making it unavailable on Windows at the moment. Could you open an issue on https://github.com/ocaml/opam-repository/issues/ instead so that the ocp-indent package is marked as available: os != "win32", as well as an issue in https://github.com/OCamlPro/ocp-indent/issues to ask them to fix this upstream?

[jacj-simcorp]

I think creating links to non-existing files works?

$ ls
$ cmd /c mklink src trgt
symbolic link created for src <<===>> trgt
$ cat src
cat: src: No such file or directory
$ echo hello world > trgt
$ cat src
hello world

I should have been clear when I said "native is no better", it does give another kind of error, but still an error:

OpamSolution.Fetch_fail("C:\\Users\\JACJ\\AppData\\Local\\Programs\\opam\\bin\\opam.exe: \"lstat\" failed on C:\\Users\\JACJ\\AppData\\Local\\Temp\\opam-28604-4fa965\\ocp-indent-1.8.1\\tests\\inplace\\link.ml: No such file or directory")

[dra27]

The underlying technical problem in Windows is slightly more subtle. It's not that you can't create symlinks to files which don't exist, it's that symlinks have types (cf. Unix.symlink docs) - i.e. a symlink to a file is different from a symlink to a directory (and a file symlink pointing to a directory doesn't work and vice versa). Cygwin's behaviour with native set is to fallback to other symlinking methods when the file doesn't exist.

I had for quite some time been debating writing+submitting a patch to GNU tar to collect symlinks to files which don't exist and then processing them at the end of the extraction (no idea whether that would have been accepted, though) but, given your other work with ocaml-tar already, we could also mitigate this by checking the tarball ourselves on extraction and dealing with any missing symlinks or we could even simply use ocaml-tar to do the extraction entirely. The only thing which had stopped me doing that in the past was that ocaml-tar had a lot of dependencies, but you've dealt with that!

[kit-ty-kate]

@jacj-simcorp A potential fix (when winsymlinks:native) was opened in #5953