Change request to support relative paths for css_reference

[maettu-this]

//css_reference should support relative paths as //css_import and //css_resource are doing.

Rationale: Referencing an own .dll located in a different project/folder than the script itself.

[oleg-shilo]

It already does. Please provide the test case if you believe that it doesn't.

[maettu-this]

My setup still uses v3.3.0 / v3.3.5.0 and I did not find information regarding relative paths in the release notes, I only found information regarding absolute paths. And, CSScript.chm of the latest version as well as the online documentation still states to very same, i.e. "the same directory where the script is". So, I assumed that there was no relative path related change with css_reference since v3.3.0.

Allow me some time to migrate from v3.3.0 to v3.19.0, I will then comment my findings.

[maettu-this]

OK, I have updated to v3.19.0, and I could track down the issue to some degree.

First I tried with a simple test DLL and script, using cscs.exe directly:
'css_reference' works fine, with child (.\Dir) as well as parent (..\Dir) relative paths. So far, so good.

However, in my application (based on CSScriptLibrary) the same thing doesn't work, it only works when the DLL is located at the very same directory as the script. My application...
...first compiles the script using CSScript.CompileFile(),...
...then instantiates an AsmHelper object,...
...and invokes the AsmHelper.

That invocation throws a System.IO.FileNotFoundException "the file or assambly ... has not been found":

• bei System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
• bei System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
• bei CSScriptLibrary.IAsmBrowser.Invoke(String methodName, Object[] list)
• bei CSScriptLibrary.AsmHelper.Invoke(String methodName, Object[] list)
• bei MyProject.ScriptExecutor.Execute() in \ScriptExecutor.cs:Zeile 160.

Two things puzzle me:

• "the file or assambly Library ..." does not state Library.dll, simply the file name without any extension ?!?

• The AsmHelper's ProbingDirs contains a weird directory (=> the last in the list):

  • "C:\Windows\Microsoft.NET\Framework64\v2.0.50727"
  • "C:\Windows\assembly\GAC_MSIL\Microsoft.VisualStudio.HostingProcess.Utilities\9.0.0.0__b03f5f7f11d50a3a"
  • "C:\Windows\assembly\GAC_MSIL\System.Windows.Forms\2.0.0.0__b77a5c561934e089"
  • "C:\Windows\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089"
  • "C:\Windows\assembly\GAC_MSIL\System.Drawing\2.0.0.0__b03f5f7f11d50a3a"
  • "C:\Windows\assembly\GAC_MSIL\Microsoft.VisualStudio.HostingProcess.Utilities.Sync\9.0.0.0__b03f5f7f11d50a3a"
  • "C:\Windows\assembly\GAC_MSIL\Microsoft.VisualStudio.Debugger.Runtime\9.0.0.0__b03f5f7f11d50a3a"
  • "C:\MyWorkspace\05-Source\Albatros\Albatros\Albatros\bin\Debug"
  • "C:\Windows\assembly\GAC_MSIL\System.Configuration\2.0.0.0__b03f5f7f11d50a3a"
  • "C:\Windows\assembly\GAC_MSIL\System.Xml\2.0.0.0__b77a5c561934e089"
  • "C:\Windows\assembly\GAC_MSIL\System.Deployment\2.0.0.0__b03f5f7f11d50a3a"
  • "C:\Windows\assembly\GAC_MSIL\System.Deployment.resources\2.0.0.0_de_b03f5f7f11d50a3a"
  • "C:\Windows\assembly\GAC_MSIL\System.Core\3.5.0.0__b77a5c561934e089"
  • "C:\Windows\assembly\GAC_MSIL\System.Xml.resources\2.0.0.0_de_b77a5c561934e089"
  • "C:\Windows\assembly\GAC_MSIL\System.Windows.Forms.resources\2.0.0.0_de_b77a5c561934e089"
  • "C:\Windows\assembly\GAC_MSIL\System.Management\2.0.0.0__b03f5f7f11d50a3a"
  • "C:\Windows\assembly\GAC_MSIL\System.resources\2.0.0.0_de_b77a5c561934e089"
  • "C:\MyWorkspace\06-Test\CSScript\css_referenceTest\Script\%CSSCRIPT_DIR%\Lib"

Why "\%CSSCRIPT_DIR%\Lib"? "\%CSSCRIPT_DIR%" might make sense (in order to be able to control the directory using //css_searchdir ?). But why adding a fixed "\Lib"? In my case, I do not define %CSSCRIPT_DIR%, so the resulting path is either "C:\MyWorkspace\06-Test\CSScript\css_referenceTest\Script\\Lib" or "C:\MyWorkspace\06-Test\CSScript\css_referenceTest\Script\Lib"?

Note that I only pass "Script.dll" to the constructor of the AsmHelper, as the help or 'asmFile' states "File name of the assembly to be loaded.". I have also tried passing the full path to the file, but that doesn't change the weird path above.

By the way, I'd personally prefer more specific variable names, such as asmFileName or asmFilePath, following the System.IO.Path terminology (i.e. FileName/Path/FullPath), and FilePath for unspecific (i.e. relative or absolute) file paths.

Does this piece of information help in some way? If you'd prefer, I could debug into AsmHelper by adding an additional configuration to my project which uses the CSScriptLibrary source code instead of the .dll. Let me know if that would help.

[maettu-this]

Update:

One of the assemblies of my project still referred to v3.3.0. I have just noticed and fixed that, now ProbingDirs contains an additional path:

• "C:\MyWorkspace\06-Test\CSScript\css_referenceTest\Script\%CSSCRIPT_DIR%\lib"
• "C:\MyWorkspace\06-Test\CSScript\css_referenceTest\Script\%CSSCRIPT_INC%"

But still, I get the System.IO.FileNotFoundException exception.

I have also searched and found information on %CSSCRIPT_DIR% in the docs. But I did not find any information on %CSSCRIPT_INC%" in either CSScript.chm nor CSScriptLibrary.chm.

[oleg-shilo]

"the file or assembly Library ..." does not state Library.dll, simply the file name without any extension ?!?

The simple answer is "because .NET runtime reports any failed assembly loading this way". It refers the failed loading with the assembly name but not the assembly file name.

---

The problem you are experiencing is caused by the inadequate assembly probing on the hosting solution.

First of of all your script compiling was OK. No problem there. After you got the compiled script path with CompileFile your scripting is done. The rest of your actions (assembly loading) has nothing to do with scripting as such. You can even load the assembly without CS-Script's AsmHelper but directly with Assembly.Load. And this is exactly what AsmHelper does, it calls Assembly.LoadFrom. AsmHelper also does optional assembly unloading but you are not using this functionality anyway.

If you indeed call Assembly.LoadFrom("script.dll") you would get the same error as if you use AsmBrowser. Your script.dll depends on library.dll, which is not in the location that Assembly.LoadFrom knows about. .NET provides the solution for the situations like this. You need to set up AppDomain.AssemblyResolve event handler and provide a hint for CLR where to find the library.dll file at runtime.

AsmHelper simplifies handling AppDomain.AssemblyResolve event by automatically probing directories from CSScript.GlobalSettings.SearchDirs:

if (CSScript.AssemblyResolvingEnabled)
    foreach (string dir in CSScript.GlobalSettings.SearchDirs.Split(';'))
        if (dir != "")
            dirs.Add(Environment.ExpandEnvironmentVariables(dir));

Thus you can do the probing by yourself or ensure that the required directory is in the global settings and let AsmHelper to do the job. Again, this is a common dynamic assembly loading challenge that has nothing to do with CS-Script. In fact CS-Script faces the same problem when the script is executed with cscs.exe. And it solves the the same way as I just described. By handling AppDomain.AssemblyResolve event.

Now you probably have a good feel why spreading the assemblies over multiple directories is not such a good idea. :)

The AsmHelper's ProbingDirs contains a weird directory...

This directory is the result of expanding %CSSCRIPT_DIR%\Lib, you already discovered that. CSSCRIPT_DIR is the environment variable that is created by the CS-Script installation.

CS-Script allows configuring the optional global probing directories via CSScript.GlobalSettings.SearchDirs. With the defaults as below:

string searchDirs = "%CSSCRIPT_DIR%\lib;%CSSCRIPT_INC%;";

You can clear the defaults by setting CSScript.GlobalSettings.SearchDirs to empty string. Though you don't have to as the script engine simply ignores any probing directory that doesn't exist.

But why adding a fixed "\Lib"?

Because it is the directory that is dedicated for system wide script dependencies (assemblies and scripts). Having them in the cs-script root directory (which is %CSSCRIPT_DIR%) isn't really practical.

As a side note, AsmHelper was developed very long time (more that 10 years) ago when there were no good way to deal with dynamically loaded assemblies. Today you have some better choices. Like dynamic. Thus your use case can be rewritten as the code below:

Script file script.cs:

//css_ref lib\child_script.dll;
using System;
using System.Windows.Forms;

public class Script
{
    static public void Main()
    {
        new MainScript().test();
    }

    public void test()
    {
        ChildScript.test();
    }
}

Script host:

CSScript.GlobalSettings.AddSearchDir(@"E:\Dev\lib");
dynamic script = CSScript.Load(@"E:\Dev\script.cs")
                         .CreateObject("*");
script.test();

I'd personally prefer more specific variable names, such as asmFileName...

It is a difficult balance between verbosity and accuracy but I do agree with you in general. I will update the signature with the next opportunity.

[oleg-shilo]

For the arguments renaming I opted to improving XML documentation as an adequate compromise.
The change 1bec072 will be available in the next release.

[maettu-this]

Hi Oleg, yesterday I updated to v3.21.1 and have again looked at this issue. Unfortunately this issue is still not solved in my opinion, and I try to express myself better than when initially describing my request.

Compilation of a script with //css_reference SomeAbsoluteOrRelativePath.SomeLibrary.dll; works fine. So far so good.

But, loading the assembly results in a System.IO.FileNotFoundException "the file or assambly ... has not been found" due to the missing probing directories for the reasons you have explained above.

While %CSSCRIPT_INC% may be a good solution for standalone / command line execution, I don't think it's suitable for a hosted application. So I am suggesting the following:

• Apparently, CSScript.CompileFile() parses the script and looks up the SomeAbsoluteOrRelativePath of //css_reference as well as //css_import, so it can find all source code and assemblies required for compilation.
• AsmHelper / .NET apparently doesn't know the SomeAbsoluteOrRelativePath when loading the assembly.

Could CSScript.CompileFile() return all paths that were used for looking up source code and assemblies? And, could AsmHelper provide a method to append probing dirs (instead of having to globally add CSScript.GlobalSettings.AddSearchDir())? Then, I could retrieve the resolved paths from CompileFile() and forward them to the AsmHelper, such that it will allow .NET to find all required assemblies. As a result, from the script user's point of view, //css_reference would then provide full support of absolute and relative paths.

Of course, I could also manually parse the script's //css_reference items and then add the paths. But this is somewhat duplicated logic, as CSScript.CompileFile() already parses these items.

[oleg-shilo]

OK, I see your point. But it's not as simple as it may seem.

Assembly resolving is a global activity. It is done within the scope of the AppDomain. Meaning that even if AsmHelper had some operation specific set of probing dirs it wouldn't be able to map tham to the AssemblyResolve events fired from different AsmHelper.Invoke calls. Nothing we can do about this as this is how CLR is implemented. That is why CS-Script is relying on global SearchDirs as a generic asm probing solution.

However nothing prevents you from attempting to implement a high resolution (non generic) assembly probing by yourself. What you need is the knowledge of the assemblies (as you already suggested) that were used during the compilation and your own AssemblyResolve handler:

AppDomain.CurrentDomain.AssemblyResolve += ResolveEventHandler;

Now, should you try to discover the assemblies by parsing the code you don't have to do it manually. Use already existing routine that does exactly that:

string[]  asms = CSScript.CodeDomEvaluator.GetReferencedAssemblies(script_code);

You can have a look at the implementation of GetReferencedAssemblies and modify it if required. But most likely it is OK as it is.

Your proposal to have some way of preserving the compiler input info (e.g. assemblies required for compilation) is also OK though there is a much better than CompileFile() candidate for that. There is a CSScript.CompilingHistory global dictionary that keeps compilation results

 public static Dictionary<FileInfo, CompilerResults> CompilingHistory;

It just makes sense to extend it and also keep CompilerParams, which is the C# compiler input that lead to the CompilerResults.

Will schedule the change for the next release.

[oleg-shilo]

Done. I will probably adjust the API a bit more but currently it can be used as this:

CSScript.KeepCompilingHistory = true;

CSScript.CompileFile(script, null, false, null);

CompilingInfo info = CSScript.CompilingHistory
                             .Values
                             .First(item => item.ScriptFile == script);

Console.WriteLine("Script: " + info.ScriptFile);

Console.WriteLine("Referenced assemblies:");
foreach (string asm in info.Input.ReferencedAssemblies)
    Console.WriteLine(asm);

if (info.Result.Errors.HasErrors)
{
    Console.WriteLine("Errors:");
    foreach (CompilerError err in info.Result.Errors)
        if (!err.IsWarning)
            Console.WriteLine("Error: " + err.ErrorText);
}

CSScript.CompilingHistory.Clear();
CSScript.KeepCompilingHistory = false;

[maettu-this]

Thanks! And thanks for the very quick v3.22! I have updated to v3.22 today but have not yet been able to adapt my code according to your directions above. I'll do that next week, but you may already close this issue as I assume it now works for me. Thanks again!

[maettu-this]

Hi Oleg, works perfectly, thanks!

I am using it as follows:

helper = new AsmHelper(assemblyFilePath, "ScriptDomain", false);

// Retrieve the distinct directories of the referenced assemblies:
csscript.CompilingInfo info = CSScript.CompilingHistory.Values.First();
var directories = new List<string>(info.Input.ReferencedAssemblies.Count);
foreach (string path in info.Input.ReferencedAssemblies)
	directories.Add(Path.GetDirectoryName(Environment.ExpandEnvironmentVariables(path)));

IEnumerable<string> referencedDirectories = directories.Distinct(PlatformSpecificPathStringComparer);

// Combine the helper's probing directories with those of the referenced assemblies:
var paths = new List<string>(helper.ProbingDirs.Count + referencedDirectories.Count());

foreach (string path in helper.ProbingDirs)
	paths.Add(Environment.ExpandEnvironmentVariables(path));

foreach (string path in referencedDirectories)
	paths.Add(Environment.ExpandEnvironmentVariables(path));

helper.ProbingDirs = paths.Distinct(PlatformSpecificPathStringComparer);

As a result, a script can now reference whatever it wants:

//css_reference .\LocalLibrary.dll;
//css_reference .\Child\ChildLibrary.dll;
//css_reference ..\ParentLibrary.dll;
//css_reference ..\Peer\PeerLibrary.dll;
//css_reference ..\Peer\Child\PeerChildLibrary.dll;

Thanks again!

Best regards,
Matthias