Supress warning SMC initialization

[aloctavodia]

This is a little bit weird. Catching the warning makes the progressbar not progress. But it seems this is not related to the warning itself, manually removing the warning from sample_prior_predictive has the same effect. Additionally it works when core=1, i.e. not parallel sampling and it also works if we add a print statement. So I guess is related to something going on here

pymc/pymc/smc/sample_smc.py

Lines 398 to 417 in 5703a9d

	def run_chains_parallel(chains, progressbar, to_run, params, random_seed, kernel_kwargs, cores):
	pbar = progress_bar((), total=100, display=progressbar)
	pbar.update(0)
	pbars = [pbar] + [None] * (chains - 1)
	pool = mp.Pool(cores)
	# "manually" (de)serialize params before/after multiprocessing
	params = tuple(cloudpickle.dumps(p) for p in params)
	kernel_kwargs = {key: cloudpickle.dumps(value) for key, value in kernel_kwargs.items()}
	results = _starmap_with_kwargs(
	pool,
	to_run,
	[(*params, random_seed[chain], chain, pbars[chain]) for chain in range(chains)],
	repeat(kernel_kwargs),
	)
	results = tuple(cloudpickle.loads(r) for r in results)
	pool.close()
	pool.join()
	return results

@ricardoV94 any guess?

[ricardoV94]

Might you be catching too many things with that context manager? Including whatever progressbar uses for piping the messages?

[aloctavodia]

It is not the catching, removing the warning has the same effect. And works as intended when no parallel sampling.

[ricardoV94]

It is not the catching, removing the warning has the same effect. And works as intended when no parallel sampling.

What about a model that would not raise the warning?

[codecov]

Codecov Report

Merging #5827 (a67b9b4) into main (6b22ed5) will decrease coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #5827      +/-   ##
==========================================
- Coverage   89.40%   89.38%   -0.02%     
==========================================
  Files          74       74              
  Lines       13772    13777       +5     
==========================================
+ Hits        12313    12315       +2     
- Misses       1459     1462       +3     

Impacted Files	Coverage Δ
pymc/smc/smc.py	96.45% <100.00%> (+0.07%)	⬆️
pymc/parallel_sampling.py	86.46% <0.00%> (-1.00%)	⬇️

[aloctavodia]

Not working. Also I just checked that is not working inside VS code or the browser. And checked a couple of older versions of fastprogressbar as well.

[ricardoV94]

I remember the code was pretty hacky / fragile and I could only see output in some contexts (e.g., terminal vs pycharm). So I am not super shocked

[aloctavodia]

What about just leaving the "print()" for the moment and open an issue so we can back to this after the release? It is possible to locally disable the pre-commit so it does not complain about the print?

[ricardoV94]

You can do sys.stdout.write(msg) I think, which will do the same as print

[ricardoV94]

I just confirmed that I see the message in a vanilla bash terminal, but not in Jupyter Notebook. This definitely used to work in notebooks:

import pymc as pm

with pm.Model() as m:
    x = pm.Normal("x")
    y = pm.Normal("y", x, observed=5)
    pm.sample_smc()

[ricardoV94]

What a hack xD!!! LGTM otherwise

[jucor]

Heads up that today this hack bit us, in an admittedly niche setup:

• I'm calling pymc
• within a python chunk
• in a RStudio Rmarkdown document thanks to the package reticulate.
• within a conda environment (pfew)
• on Windows 10 😆 (The problem does not appear on my Mac machine, only on my colleague @CarolinePLX Win10 machine)

So it is interactive and called from something else than a terminal or a jupyter notebook, and on a Windows machine to boot.

And in that case, apparently stdout has been somewhere set to None, as I get the error nonetype object has no attribute write 😢

I'm heartbroken 💔 and super tickled of trying to find out what's going on. I'll do more debugging next week (need to dig up another Windows machine for my own tests), will report here. Maybe I'll simply add a check that stdout isn't None before deploying the hack? (a hack on a hack 😆 )

In the meantime if you have any idea, they're very welcome!

[jucor]

Yup, I can confirm that the below patch fixes the issue with reticulate (which indeed seems to mangle somewhat stdout). At the moment I'm monkey-patching the package on the fly (🐷 ). Want me to open a proper PR?

--- a/kernels.py	2024-06-12 22:15:14.506303300 +0200
+++ b/kernels.py	2024-06-12 22:15:32.926642800 +0200
@@ -186,7 +186,8 @@
 
     def initialize_population(self) -> dict[str, np.ndarray]:
         """Create an initial population from the prior distribution"""
-        sys.stdout.write(" ")  # see issue #5828
+        if sys.stdout is not None:
+          sys.stdout.write(" ")  # see issue #5828
         with warnings.catch_warnings():
             warnings.filterwarnings(
                 "ignore", category=UserWarning, message="The effect of Potentials"