Improve IncludeTestAssembly

[tonyhallett]

Coverlet IncludeTestAssembly does not guarantee coverage for test assembly

This flag / runsettings setting only ensures that the module is considered.

Coverage class

    public CoveragePrepareResult PrepareModules()
    {
      string[] modules = _instrumentationHelper.GetCoverableModules(_moduleOrAppDirectory, _parameters.IncludeDirectories, _parameters.IncludeTestAssembly);
...
      _parameters.ExcludeFilters = _parameters.ExcludeFilters?.Where(f => _instrumentationHelper.IsValidFilterExpression(f)).ToArray();
      _parameters.IncludeFilters = _parameters.IncludeFilters?.Where(f => _instrumentationHelper.IsValidFilterExpression(f)).ToArray();

      IReadOnlyList<string> validModules = [.. _instrumentationHelper.SelectModules(modules, _parameters.IncludeFilters, _parameters.ExcludeFilters)];

Then the filters are applied
filters docs

Both Exclude and Include properties can be used together but Exclude takes precedence.

From below can see that is Includes except Excludes

InstrumentationHelper

    public IEnumerable<string> SelectModules(IEnumerable<string> modules, string[] includeFilters, string[] excludeFilters)
    {
      const char escapeSymbol = '!';
      ILookup<string, string> modulesLookup = modules.Where(x => x != null)
          .ToLookup(x => $"{escapeSymbol}{Path.GetFileNameWithoutExtension(x)}{escapeSymbol}");

      string moduleKeys = string.Join(Environment.NewLine, modulesLookup.Select(x => x.Key));
      string includedModuleKeys = GetModuleKeysForIncludeFilters(includeFilters, escapeSymbol, moduleKeys);
      string excludedModuleKeys = GetModuleKeysForExcludeFilters(excludeFilters, escapeSymbol, includedModuleKeys);

      IEnumerable<string> moduleKeysToInclude = includedModuleKeys
          .Split([Environment.NewLine], StringSplitOptions.RemoveEmptyEntries)
          .Except(excludedModuleKeys.Split([Environment.NewLine], StringSplitOptions.RemoveEmptyEntries));

      return moduleKeysToInclude.SelectMany(x => modulesLookup[x]);
    }

** Note that as soon as an Include is present to be included there needs to be an include **

    private string GetModuleKeysForIncludeFilters(IEnumerable<string> filters, char escapeSymbol, string moduleKeys)
    {
      string[] validFilters = GetValidFilters(filters);

      return validFilters.Length == 0 ? moduleKeys : GetIncludeModuleKeysForValidFilters(escapeSymbol, moduleKeys, validFilters);
    }

InstrumentationHelperTests

So without include filters IncludeTestAssembly will succeed. Whereas if the FCC setting IncludeReferencedProjects is used it will not.

FCC can add the test assembly to the includeFilters in the following code

FineCodeCoverage/SharedProject/Core/Coverlet/DataCollector/DataCollectorSettingsBuilder.cs

Line 132 in 3673239

private XElement GenerateDataCollectorElement()

FineCodeCoverage/SharedProject/Core/Coverlet/Console/CoverletConsoleUtil.cs

Line 46 in 3673239

foreach (var value in (project.Settings.Include ?? new string[0]).Where(x => !string.IsNullOrWhiteSpace(x)))

Although by doing so will then be excluding the SUT unless that has been included manually or via IncludeReferencedProjects !
** solution** is to only include the test assembly in the includes when necessary.

Also, if desired, the environment variable
COVERLET_DATACOLLECTOR_OUTOFPROC_DEBUG
will launch a debugger.

ms code coverage behaves similarly......................
Docs

You can include or exclude assemblies or specific types and members from code coverage analysis. If the Include section is empty or omitted, then all assemblies that are loaded and have associated PDB files are included. If an assembly or member matches a clause in the Exclude section, then it is excluded from code coverage. The Exclude section takes precedence over the Include section: if an assembly is listed in both Include and Exclude, it will not be included in code coverage.

Note that IncludeTestAssembly is by default true. If it is false then will be added to ModulePaths.Excluded.

From ms code coverage 17.13

ModuleList ( module paths ) gets its behaviour from its base class ( as they all do )

// Decompiled with JetBrains decompiler
// Type: Microsoft.CodeCoverage.Core.Configurations.ExcludeIncludeList
// Assembly: Microsoft.CodeCoverage.Core, Version=17.13.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
// MVID: FEEBC821-D77A-43A4-9166-2C626CD4A0FE
// Assembly location: C:\Users\tonyh\AppData\Local\FineCodeCoverage\msCodeCoverage\17.13.0\build\netstandard2.0\Microsoft.CodeCoverage.Core.dll

using Microsoft.CodeCoverage.Core.Utils;
using System;
using System.Collections;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Collections.Specialized;
using System.Linq;
using System.Xml.Linq;


#nullable enable
namespace Microsoft.CodeCoverage.Core.Configurations
{
  [Serializable]
  internal abstract class ExcludeIncludeList
  {
    private const bool MergeDefaultsDefault = true;
    private const string MergeDefaultsAttributeName = "mergeDefaults";
    private const string IncludeElementName = "Include";
    private const string ExcludeElementName = "Exclude";
    private readonly IList<RegularExpression> _includedExpressions = (IList<RegularExpression>) new List<RegularExpression>();
    private readonly IList<RegularExpression> _excludedExpressions = (IList<RegularExpression>) new List<RegularExpression>();
    private bool _includeAll = true;
    private bool _hasInclude;
    private bool _excludeAll;

    public ExcludeIncludeList(XElement? element = null)
    {
      this.Excluded = new ObservableCollection<string>();
      this.Included = new ObservableCollection<string>();
      this.Excluded.CollectionChanged += new NotifyCollectionChangedEventHandler(this.Excluded_CollectionChanged);
      this.Included.CollectionChanged += new NotifyCollectionChangedEventHandler(this.Included_CollectionChanged);
      if (element == null)
        return;
      this.MergeDefaults = XmlUtils.ReadAttributeBoolValue(element, "mergeDefaults", true);
      XElement xelement1 = element.Element((XName) "Include");
      if (xelement1 != null)
      {
        foreach (string str in xelement1.Elements().Select<XElement, string>((Func<XElement, string>) (e => e.Value)))
          this.Included.Add(str);
      }
      XElement xelement2 = element.Element((XName) "Exclude");
      if (xelement2 == null)
        return;
      foreach (string str in xelement2.Elements().Select<XElement, string>((Func<XElement, string>) (e => e.Value)))
        this.Excluded.Add(str);
    }

    public bool MergeDefaults { get; set; } = true;

    private void Included_CollectionChanged(object? _, NotifyCollectionChangedEventArgs e)
    {
      if (!this._hasInclude)
      {
        this._includeAll = false;
        this._hasInclude = true;
      }
      this.UpdateExpressionsList(e, this._includedExpressions, ref this._includeAll);
    }

    private void Excluded_CollectionChanged(object? _, NotifyCollectionChangedEventArgs e) => this.UpdateExpressionsList(e, this._excludedExpressions, ref this._excludeAll);

    private void UpdateExpressionsList(
      NotifyCollectionChangedEventArgs e,
      IList<RegularExpression> expressions,
      ref bool all)
    {
      if (all || e.Action != NotifyCollectionChangedAction.Add || e.NewItems == null)
        return;
      foreach (string newItem in (IEnumerable) e.NewItems)
      {
        if (newItem == "*" || newItem == ".*")
        {
          all = true;
          expressions.Clear();
          break;
        }
        if (!newItem.Contains("System.CodeDom.Compiler.GeneratedCodeAttribute") && !newItem.Contains("System.Runtime.CompilerServices.CompilerGeneratedAttribute"))
          expressions.Add(new RegularExpression(newItem, this.IgnoreCase, this.IgnorePathSeparator));
      }
    }

    public ObservableCollection<string> Excluded { get; set; }

    public ObservableCollection<string> Included { get; set; }

    protected abstract string ElementName { get; }

    protected abstract bool IgnoreCase { get; }

    protected abstract bool IgnorePathSeparator { get; }

    public bool IncludeAll => this._includeAll && this.Excluded.Count == 0;

    public virtual bool IsIncluded(string input)
    {
      if (input == null)
        return true;
      return this._includedExpressions.Count == 0 && this._excludedExpressions.Count == 0 ? this._includeAll && !this._excludeAll : this.InternalIsIncluded(input) && !this.InternalIsExcluded(input);
    }

    public bool IsIncludedExplicitly(string input) => this.MatchesExpression(input, this._includedExpressions) && !this.InternalIsExcluded(input);

    private bool InternalIsIncluded(string input)
    {
      if (this._includeAll)
        return true;
      return this._includedExpressions.Count != 0 && this.MatchesExpression(input, this._includedExpressions);
    }

    private bool InternalIsExcluded(string input)
    {
      if (this._excludeAll)
        return true;
      return this._excludedExpressions.Count != 0 && this.MatchesExpression(input, this._excludedExpressions);
    }

    private bool MatchesExpression(string input, IList<RegularExpression> expressions)
    {
      foreach (RegularExpression expression in (IEnumerable<RegularExpression>) expressions)
      {
        if (expression.IsValid && expression.Match(input))
          return true;
      }
      return false;
    }

..
  }
}

OpenCover

Program class will add the default include all filter if no filters

        private static IFilter BuildFilter(CommandLineParser parser)
        {
            var filter = Filter.BuildFilter(parser);
            if (!string.IsNullOrWhiteSpace(parser.FilterFile))
            {
                if (!File.Exists(parser.FilterFile.Trim()))
                    System.Console.WriteLine("FilterFile '{0}' cannot be found - have you specified your arguments correctly?", parser.FilterFile);
                else
                {
                    var filters = File.ReadAllLines(parser.FilterFile);
                    filters.ToList().ForEach(filter.AddFilter);
                }
            }
            else
            {
                if (parser.Filters.Count == 0)
                    filter.AddFilter("+[*]*");
            }

            return filter;
        }

Determination from Filter class

        public bool UseAssembly(string processName, string assemblyName)
        {
            if (ExcludeProcessOrAssembly(processName, assemblyName, out IList<AssemblyAndClassFilter> matchingExclusionFilters))
                return false;

            if (matchingExclusionFilters.Any(exclusionFilter => exclusionFilter.ClassName != ".*"))
                return true;

            var matchingInclusionFilters = InclusionFilters.GetMatchingFiltersForAssemblyName(assemblyName);

            return matchingInclusionFilters.Any();
        }

        private bool ExcludeProcessOrAssembly(string processName, string assemblyName, out IList<AssemblyAndClassFilter> matchingExclusionFilters)
        {
            matchingExclusionFilters = ExclusionFilters.GetMatchingFiltersForAssemblyName(assemblyName);
            return matchingExclusionFilters.Any(
                exclusionFilter =>
                    exclusionFilter.ClassName == ".*" &&
                    (exclusionFilter.IsMatchingProcessName(processName)));
        }

[tonyhallett]

released