Composer cannot find the drupal-scaffold command in post-install.sh

[chriscohen]

I have tweaked the DOCUMENTROOT variable in post-install.sh to be something else because "web" won't work for us as a directory name.

I have also tweaked the "drupal-composer/drupal-scaffold" to use "dev-use-better-apis" because of the apparent bug preventing the main branch from working at the moment.

Everything seems to download fine in composer, but then the post-install.sh script has issues straight away.

The vendor dir definitely contains the class in question, but I don't know enough about this side of composer to know why it apparently cannot find the drupal-scaffold command as defined.

> sh ./scripts/composer/post-install.sh
./scripts/composer/post-install.sh: line 8: composer: command not found
cp: cannot stat `docroot/sites/default/default.settings.php': No such file or di
rectory
chmod: getting attributes of `docroot/sites/default/settings.php': No such file
or directory
Create a sites/default/settings.php file with chmod 666
cp: cannot stat `docroot/sites/default/default.services.yml': No such file or di
rectory
chmod: getting attributes of `docroot/sites/default/services.yml': No such file
or directory
Create a sites/default/services.yml services.yml file with chmod 666
mkdir: cannot create directory `docroot/sites/default/files': No such file or di
rectory
chmod: getting attributes of `docroot/sites/default/files': No such file or dire
ctory
Create a sites/default/files directory with chmod 777

[greg-1-anderson]

So, your problem is not specifically drupal-scaffold, but more generally the composer executable itself:

./scripts/composer/post-install.sh: line 8: composer: command not found

Do you have composer on your global $PATH, or are you perhaps running some variant of php composer.phar? The simplest solution would be to add composer to your $PATH. At the moment, drupal-project does not require composer itself; if it did (or if we added it), then we could guarantee that we could always run the local composer. I don't think that we should do that by default, but perhaps if the global composer is not found, we could look for a local composer. That way, folks could always add composer to their own project if they preferred to do it that way.

[greg-1-anderson]

Side note: it should no longer be necessary to adjust the version/branch of drupal-scaffold in your composer.json file. Run composer update, and ensure that you have drupal-scaffold 1.1.1 and codegyre/robo 0.7.0, and that should clear up the bugs with cross-volume renames, and spaces in filenames.

It will still be necessary to have composer in your $PATH.

[chriscohen]

I don't think this is related to having composer in my $PATH. I'm using Git Bash to run ./scripts/composer/post-install.sh. If I run the command manually, it does the same thing with the "composer: command not found" error.

If I run "composer" manually, it works fine. I'm not sure what else is going on here.

composer drupal-scaffold just doesn't work on Windows... if I run manually:

$ composer drupal-scaffold
> DrupalComposer\DrupalScaffold\Plugin::scaffold
'"C:\Webs\mysite\vendor/codegyre/robo/robo"' is not recognized as an internal
or external command,
operable program or batch file.

PS I've updated back to ~1 for drupal-scaffold and it's getting 1.1.1 with robo 0.7.0.

[greg-1-anderson]

I think that if we used the PHP curl API to download files instead of Guzzle, then we could call Robo directly rather than exec'ing it. This would probably be the best way to solve this problem.

I am curious, though: if you manually run C:\Webs\mysite\vendor\codegyre\robo\robo test, does it work? Might just be a path separator error.

[chriscohen]

In Git Bash, this works (using forward slash path separators. In MS-DOS this does not work. It says "is not recognised as an internal or external command".

Ideally it would work either way, but I think it's that Windows doesn't understand "executable" files because it's Windows, so another method might be required.

[greg-1-anderson]

Please try with drupal-composer/drupal-scaffold#26 and see if that clears up your problems on Windows.

[chriscohen]

Sorry. I don't know what you mean by that. What do I need to change to use that branch or whatever it is?

[greg-1-anderson]

In the require section of your composer.json, find the drupal-composer/drupal-scaffold line, and change it as follows:
"drupal-composer/drupal-scaffold": "dev-curl-download",

Then, run composer update followed by composer drupal-scaffold, and let me know if the drupal-scaffold command works on Windows now.

[chriscohen]

Okay, looks like that worked. It didn't fix it completely, but here's my latest output:

$ composer drupal-scaffold
> DrupalComposer\DrupalScaffold\Plugin::scaffold
 [FileSystem\FilesystemStack] mkdir ["C:\\Users\\ChrisC\\AppData\\Local\\Temp\\\
/drupal-scaffold-1456239029"]
 [FileSystem\FilesystemStack] _chmod ["C:\\Webs\\mysite\\docroot\/sites\/defau
lt",493]
 [FileSystem\CleanDir] Cleaned C:\Users\ChrisC\AppData\Local\Temp\/drupal-scaffo
ld-1456239029
>  Attempt to download https://ftp.drupal.org/files/projects/drupal-8.0.3.tar.gz
 to C:\Users\ChrisC\AppData\Local\Temp\/drupal-scaffold-1456239029/drupal-8.0.3.
tar.gz

 [Archive\Extract] Extracting C:\Users\ChrisC\AppData\Local\Temp\/drupal-scaffol
d-1456239029/drupal-8.0.3.tar.gz
PHP Warning:  require_once(PEAR.php): failed to open stream: No such file or dir
ectory in C:\Webs\mysite\vendor\pear\archive_tar\Archive\Tar.php on line 42

Warning: require_once(PEAR.php): failed to open stream: No such file or director
y in C:\Webs\mysite\vendor\pear\archive_tar\Archive\Tar.php on line 42
PHP Fatal error:  require_once(): Failed opening required 'PEAR.php' (include_pa
th='.;C:\Program Files (x86)\DevDesktop\common\pear') in C:\Webs\mysite\vendor
\pear\archive_tar\Archive\Tar.php on line 42

Fatal error: require_once(): Failed opening required 'PEAR.php' (include_path='.
;C:\Program Files (x86)\DevDesktop\common\pear') in C:\Webs\mysite\vendor\pear
\archive_tar\Archive\Tar.php on line 42
ERROR: require_once(): Failed opening required 'PEAR.php' (include_path='.;C:\Pr
ogram Files (x86)\DevDesktop\common\pear')
in C:\Webs\mysite\vendor\pear\archive_tar\Archive\Tar.php:42

[greg-1-anderson]

Looks like PEAR is not installed on your system. Not sure if this is installed by default on Linux / Mac, or merely widely installed. We could fall back to using the tar/zip executables if PEAR is not available, perhaps.

Another thing to test would be to use the zip archive of Drupal rather than the tar.gz. Add this to your composer.json:

    "extra": {
        "drupal-scaffold": {
            "source": "https://ftp.drupal.org/files/projects/drupal-{version}.zip"
        }
    }

You'll need to merge this with any existing extra section. Make sure that your .json file remains valid (commas always appear on items in lists, except for the last item which cannot have a comma).

That might work.

UPDATE: Corrected URL above.

[greg-1-anderson]

Wait, the URL above is wrong. One moment while I look up the right one. Corrected URL above.

[chriscohen]

BTW PEAR is installed via Acquia Dev Desktop on this machine. I'm not sure but it's something to do with the fact that Windows is terrible and on every other OS there's a standard way of putting PEAR on your system, but you have to be really fiddly to even get it running on Windows.

[greg-1-anderson]

Instructions on installing PEAR: https://pear.php.net/manual/en/installation.getting.php

Ideally, it should not be necessary to install additional software to use drupal-composer/drupal-project, but you might wish to try this as an interim measure.

[bojanz]

Relying on PEAR sounds like a recipe for pain, maybe we can find an Archive package on Packagist and use that instead?

[greg-1-anderson]

Previously we were using zippy; this uses executables, wheras the archive package we are using now is based entirely on php code. Maybe we need both. I do not think there is an archive package available that does not rely on any external component.

[greg-1-anderson]

Actually, the problem is not that PEAR is missing; it is available via Composer. The problem we have is that PEAR uses include-path in its composer.json file, which is not available to custom installers such as drupal-scaffold.

I'm not sure why this isn't an issue on other platforms; I will investigate further.

[greg-1-anderson]

Okay, I found a solution, but it will require a change in Archive_Tar. The problem is that Archive_Tar does a require_once PEAR.php;, so that it will work on non-Composer systems. On systems that already have PEAR installed, this file is already included, so this is a no-op. On ordinary Composer-based applications (on systems without PEAR installed), the include-path directive in Archive_Tar's composer.json ensures that this file can be required. However, as I mentioned above, Composer does not respect include-path in custom installers, so the require_once fails. However, on Composer systems, the require is unnecessary, because the PEAR class is PSR-0 autoloadable.

I did a test on my system that involved renaming PEAR to PEAR2, to simulate a system that did not have PEAR installed, and it seems like my solution should work fine. It would be helpful if someone with a Windows system (maybe @chriscohen?) could also run a test for me, to confirm whether this does in fact work in their environment.

1. Open vendor/pear/archive_tar/Archive/Tar.php
2. Find the require_once 'PEAR.php'; line, and modify it as shown below:

if (!class_exists('PEAR')) {
    require_once 'PEAR.php';
}

1. Run composer drupal-scaffold

If this works, then I will submit a PR with explanation to the Archive_Tar project.

It would also be very helpful if someone could test to see if zip files work better, per the instructions above. If this works, then I can simply change the default URL in drupal-scaffold to point at the zip file, which would serve as a reasonable workaround until such a time as the Archive_Tar fix was merged in to the upstream.

[chriscohen]

That does work fine for the extraction process! Thanks for that suggestion.

The next problem is that Windows does not have rsync available (derp).

$ composer drupal-scaffold
> DrupalComposer\DrupalScaffold\Plugin::scaffold
 [FileSystem\FilesystemStack] mkdir ["C:\\Users\\ChrisC\\AppData\\Local\\Temp\\\
/drupal-scaffold-1456306227"]
 [FileSystem\FilesystemStack] _chmod ["C:\\Webs\\mysite\\docroot\/sites\/defau
lt",493]
 [FileSystem\CleanDir] Cleaned C:\Users\ChrisC\AppData\Local\Temp\/drupal-scaffo
ld-1456306227
>  Attempt to download https://ftp.drupal.org/files/projects/drupal-8.0.3.tar.gz
 to C:\Users\ChrisC\AppData\Local\Temp\/drupal-scaffold-1456306227/drupal-8.0.3.
tar.gz

 [Archive\Extract] Extracting C:\Users\ChrisC\AppData\Local\Temp\/drupal-scaffol
d-1456306227/drupal-8.0.3.tar.gz
 [Archive\Extract] C:\Users\ChrisC\AppData\Local\Temp\/drupal-scaffold-145630622
7/drupal-8.0.3.tar.gz extracted
 [FileSystem\FilesystemStack] _rename ["C:\\Webs\\mysite\/tmp239771456306229\/
drupal-8.0.3","C:\\Users\\ChrisC\\AppData\\Local\\Temp\\\/drupal-scaffold-145630
6227\/drupal-8"]
 [Remote\Rsync] Running rsync -a -v -z --include "sites" --include "sites/defaul
t" --include "sites/default/default.settings.php" --include "sites/default/defau
lt.services.yml" --include "sites/development.services.yml" --include "sites/exa
mple.settings.local.php" --include "sites/example.sites.php" --exclude ".gitkeep
" --exclude "autoload.php" --exclude "composer.json" --exclude "composer.lock" -
-exclude "core" --exclude "drush" --exclude "example.gitignore" --exclude "LICEN
SE.txt" --exclude "README.txt" --exclude "vendor" --exclude "themes" --exclude "
profiles" --exclude "modules" --exclude "sites/*" --exclude "sites/default/*" "C
:\Users\ChrisC\AppData\Local\Temp\/drupal-scaffold-1456306227/drupal-8/" "C:\Web
s\mysite\docroot"
'rsync' is not recognized as an internal or external command,
operable program or batch file.
 [Remote\Rsync]  Exit code 1  Time 0.03s
 [Robo\Result] Stopping on fail. Exiting....
 [Robo\Result] Exit Code: 1

[greg-1-anderson]

@chriscohen: Great, thanks; I'll put in a PR with the Archive_Tar project. Could you also test .zip files (without the Archive_Tar modification) and let me know if that works as well?

rsync is a very convenient way to overlay the extracted files over the Drupal project; looks like we will need a php overlay function for the convenience of Windows users. I won't be able to get to that until next week, or maybe the week after; if anyone would care to submit an OverlayDir variant of CopyDir to the Robo project, that would be very helpful. I can code that up pretty quickly myself once I have time, though.

[greg-1-anderson]

My PR in Archive_Tar has already been merged, and a new version was tagged, so now we are just down to needing a replacement for rsync.

[Lowell20]

I am getting a similar error but from a ^M character (windows line ending char) in the post-install.sh script.
If I change the line endings to unix the script runs but I get different error messages
In the end my web directory is not fully populated and there is no sites directory

I am installing on an ubuntu server

[greg-1-anderson]

@Lowell20: The problem you are having is similar but unrelated to the problem being tracked in this issue. Could you please open a new report, and include more information on the errors you are seeing?

[webflo]

@Lowell20 scripts/composer/post-install.sh has UNIX-style line endings. Maybe the issue is related to your git config?

[Lowell20]

lol, hey thanks! that's exactly what I figured out :-)
solved with
git config --global core.autocrlf input

[webflo]

@chriscohen Is it still an issue on Windows? Or do you have any workarounds?

[greg-1-anderson]

I believe that everything is working on Windows if rsync is installed; however, many Windows users do not have rsync, so it would be good to use a native PHP equivalent here instead.

[webflo]

#128 should resolve the original bug report