Monkeypatch a vendor package that is not available when running tests

[wigging]

I am using a Python package that is provided by a vendor that makes laboratory equipment. The vendor package provides an interface to control an instrument in the lab. Due to licensing, the vendor package is only on the computer that the instrument is connected to.

I made another Python package that relies on the vender package for some functionality. However, the computer where the vendor package resides is not always accessible which hampers the development of my own package. So the only time I can develop my package is when I have access to the lab computer.

I tried to monkeypatch functions from the vendor package. But it doesn't work because pytest complains that the vendor package can't be found. Below is some code to demonstrate the problem.

# This represents a module in `mypackage`. The imported `calc` package is a
# vendor package that is not available when I run the tests.

import calc  # <-- represents third-party vendor package

def divider(x: float, y: float) -> float:
    """Divide two numbers."""
    z = calc.divide(x, y)
    return z

# This test fails because pytest cannot find the vendor package which the
# `mypackage` relies on.

import mypackage


def test_divider(monkeypatch):
    """Test divider function."""

    def mock_divider(a, b):
        return 0.8

    monkeypatch.setattr(mypackage, "divider", mock_divider)

    z = mypackage.divider(9, 3)
    assert z == 0.8

Is there a way I can use monkeypatch or some other feature of pytest to run tests for my package when the vendor package is not available? I basically want to mock the vendor package so that I can develop my own package when I don't have access to the computer where the vendor package resides.

[RonnyPfannschmidt]

Create a own package that fakes the import names

Also reach out to the vendor about test automation

Some venors provide extra license specifically for development or ci

[wigging]

You suggested in a previous comment that I should create a wrapper API that I can mock/control instead of trying to mock the vendor API. That is what I did in the example in my previous comment. The wrapper API is the CalcAPI class which I mock in the test_divider function. So in the example, I mock the CalcAPI class instead of the vendor package which is represented by the calc package.

[webknjaz]

@wigging I think it makes sense more or less. But here are a few things to consider: you might want to wrap your vendored dist import into a try/except. And also, you might even not need mocking with such a design in the first place — just make a fake class in tests and initialize it. Unless you're also going to spy on that.

Though, as Ronny said, the example is rather artificial so it's hard to advise fully. Like if you're going to reimplement bits of the third-party thing logic in your tests, why don't you implement them in your code too instead of having a dependency (that's lambda x, y: x / y I'm talking about). Otherwise, just use mocking without implementing things or with hardcoding values.

On a related note, use pytest-mock which provides a nicer UX for interacting with unittest.mock.

P.S. (idiomatic Python comment) I hope you don't call your methods with nouns in the real thing.

---

Looking more, I think that having two different interfaces with adder(x: float, y: float) -> float vs. divider(calcapi: CalcAPI, x: float, y: float) -> float is probably not the best idea. You could make calcapi optional with something like *, calcapi: CalcAPI | None = None. But a better solution would be having some sort of container class with all the methods of your API. Like class Calculator: with def __init__(self, calcapi: CalcAPI | None = None) -> None: that would store the third-party interface on the object attribute, and you'd have any methods like divide() access it via smth like self._ext_calc_api internally.

And so it tests, you'd use the thing like Calculator(FakeCalcAPI()).divide(...). You could probably move Calculator(FakeCalcAPI()) into a fixture to reuse across tests and not care about the fake most of the time, while working with a uniform API.

But again, this is all very artificial. YMMW.

[wigging]

@webknjaz I tried to come up with a simple and short example for the discussion instead of using the actual code I'm working on. But I guess it's hard to offer advice for such a simple code. I'll see if I can come up with a better example. Also, what is pytest-mock, is it another pytest package or extension?

[webknjaz]

It's a pytest plugin.

[RonnyPfannschmidt]

If the vendors api is very small in turn then the value add of the patterns we propose may not manifest

As we lack background knowledge of that detail we have a communication issue there

[wigging]

Here is a more complete example. This also accounts for some of the suggestions from @RonnyPfannschmidt and @webknjaz in the other answer.

Vendor package

For demonstration purposes, here is a vendor package named arithmetic. It contains some functions to do arithmetic. This package may not be available on the computer where the tests are run. To keep things short, I'm only showing the divide function, the multiply function is similar.

vendor/
└── arithmetic/
    ├── README.md
    ├── pyproject.toml
    └── src/
        └── arithmetic/
            ├── __init__.py
            ├── divide.py
            └── multiply.py

# arithmetic/__init__.py

from .divide import divide
from .multiply import multiply

__all__ = ["divide", "multiply"]

# arithmetic/divide.py

def divide(a: float, b: float) -> float:
    """Divide two numbers."""
    z = a / b
    return z

My package

I have a calculator package that relies on the vendor arithmetic package to perform division. The file structure of this package is shown below.

calculator/
├── README.md
├── pyproject.toml
├── src/
│   └── calculator/
│       ├── __init__.py
│       ├── arithmetic_api.py
│       └── calculator.py
├── tests/
│   ├── test_addition.py
│   └── test_division.py
└── uv.lock

# src/calculator/__init__.py

from .arithmetic_api import ArithmeticAPI
from .calculator import Calculator

__all__ = ["ArithmeticAPI", "Calculator"]

# src/calculator/arithmetic_api.py

class ArithmeticAPI:
    """Wrapper API for the vendor arithmetic package."""

    def __init__(self, arithmetic) -> None:
        """Initialize with the vendor's arithmetic package."""
        self._arithmetic = arithmetic

    def divide(self, x: float, y: float) -> float:
        """Divide two numbers using the vendor arithmetic package."""
        z = self._arithmetic.divide(x, y)
        return z

# src/calculator/calculator.py

from .arithmetic_api import ArithmeticAPI

class Calculator:
    """Calculator model for arithmetic operations."""

    def __init__(self, api: ArithmeticAPI | None = None) -> None:
        self.api = api

    @staticmethod
    def add(x: float, y: float) -> float:
        """Add two numbers."""
        z = x + y
        return z

    def divide(self, x: float, y: float) -> float:
        """Divide two numbers. Uses vendor arithmetic package if available."""
        if self.api:
            print("Using the vendor arithmetic package via ArithmeticAPI")
            z = self.api.divide(x, y)
        else:
            z = x / y

        return z

Usage

This example uses the calculator package to add two numbers. It does not rely on the vendor package.

from calculator import Calculator

calc = Calculator()
z = calc.add(3, 1.5)

This example divides two numbers without using the vendor package.

from calculator import Calculator

calc = Calculator()
z = calc.divide(10, 3)

This divides two numbers using the vendor's arithmetic package via the ArithmeticAPI class. Notice the path to the arithmetic package must be added to the PYTHONPATH to run the example.

# Add the arithmetic package to path before running
# export PYTHONPATH="/Users/homer/Desktop/vendor/arithmetic/src:$PYTHONPATH"

import arithmetic  # this is the vendor's arithmetic package

from calculator import ArithmeticAPI
from calculator import Calculator

api = ArithmeticAPI(arithmetic)

calc = Calculator(api)
z = calc.divide(10, 3)

Testing

Testing the addition function is straightforward as it doesn't rely on the vendor package.

from calculator import Calculator

def test_addition():
    """Test add function."""
    calc = Calculator()
    z = calc.add(5, 12.1)
    assert z == 17.1

Testing the division function involves two tests. One test for testing without the vendor package. Another test that mocks the wrapper API as if the divide function from the vendor package was used.

from calculator import Calculator
from calculator import ArithmeticAPI
from unittest.mock import Mock

def test_division():
    """Test divide function."""
    calc = Calculator()
    z = calc.divide(9, 3)
    assert z == 3

def test_division_vendor():
    """Test divide function using vendor arithmetic package via ArithmeticAPI."""

    api = Mock(spec_set=ArithmeticAPI, divide=lambda x, y: 5.5)

    calc = Calculator(api)
    z = calc.divide(9, 3)
    assert z == 5.5

Summary

Hopefully this is a better example. I know it may not give all the details needed to provide a complete answer but I think it closely resembles the actual code I'm working with. @RonnyPfannschmidt and @webknjaz Let me know if you have more suggestions and thanks for the help.

[RonnyPfannschmidt]

Either the choosen example we don't have a chance for a suggestion

[webknjaz]

Yeah, I'm not sure how to advise further either. Although, I must note that my previous suggestion to have def __init__(self, api: ArithmeticAPI | None = None) -> None: was a bit misleading, probably because it made you think that you needed to make runtime code conditional while I meant that the calling side should inject a fake when appropriate. So it's better for it to be just def __init__(self, api: ArithmeticAPI) -> None:. And the both the tests and other runtime code must supply something there that conforms to the shape of said API.

On a related note, you should probably integrate https://pytest-cov.rtfd.io first thing and learn to use the HTML report to identify (un-)covered lines. That way, when you get to using https://pytest-mock.rtfd.io, you'll be able to visually confirm what you're testing. With such a visual clue, you'll probably gain a better understanding of how you want to test things and maybe finally throw mock out the window where it's harmful.

[wigging]

@webknjaz If I do the following then I can't use the add function without the API even though that function doesn't rely on the vendor package. But I guess what you are saying is the Calculator model here is essentially broken if the API is not given therefore it should require the ArithmeticAPI class.

from .arithmetic_api import ArithmeticAPI

class Calculator:

    def __init__(self, api: ArithmeticAPI) -> None:
        self.api = api

    @staticmethod
    def add(x: float, y: float) -> float:
        z = x + y
        return z

    def divide(self, x: float, y: float) -> float:
        z = self.api.divide(x, y)
        return z

[webknjaz]

I was originally thinking

    def __init__(self, api: ArithmeticAPI | None = None) -> None:
        self.api = api if api is not None else ArithmeticAPI()

That way, you'd supply something fake in tests. But then again, you could keep def __init__(self, api: ArithmeticAPI) -> None: and have the calling code decide what to pass, which could be fake in tests. If you want to let the users work with a semi-working thing, you could also allow None, of course. Or you could even have a fake object in runtime in some cases and gate non-working paths somehow. You may want to use typing.Protocol to decouple interfaces from (possibly missing) implementation.