Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memory error #4

Open
animesh opened this issue Mar 4, 2025 · 0 comments
Open

memory error #4

animesh opened this issue Mar 4, 2025 · 0 comments

Comments

@animesh
Copy link

animesh commented Mar 4, 2025

trying to run mza over timsTOF-pro DDA but facing following memory allocation issues

Z:\Download\mza_v1.25.02.19>mza.exe -f c:\Users\animeshs\IDH1\220202_Sigrid_28b_Slot2-28_1_905.d
Processing file: c:\Users\animeshs\IDH1\220202_Sigrid_28b_Slot2-28_1_905.d
Process Process-7:
Traceback (most recent call last):
  File "multiprocessing\process.py", line 297, in _bootstrap
  File "multiprocessing\process.py", line 99, in run
  File "source\BrukerTimsTof2mza.py", line 37, in parse_partition
  File "pyTDFSDK\classes.py", line 102, in __init__
  File "pyTDFSDK\classes.py", line 125, in get_db_tables
  File "pyTDFSDK\classes.py", line 125, in <dictcomp>
  File "pandas\io\sql.py", line 443, in read_sql_query
  File "pandas\io\sql.py", line 2139, in read_query
  File "pandas\io\sql.py", line 172, in _wrap_result
  File "pandas\core\frame.py", line 2073, in from_records
  File "pandas\core\internals\construction.py", line 797, in to_arrays
  File "pandas\core\internals\construction.py", line 815, in _list_to_arrays
  File "pandas\_libs\lib.pyx", line 2955, in pandas._libs.lib.to_object_array_tuples
MemoryError
Process Process-5:
Traceback (most recent call last):
  File "multiprocessing\process.py", line 297, in _bootstrap
  File "multiprocessing\process.py", line 99, in run
  File "source\BrukerTimsTof2mza.py", line 37, in parse_partition
  File "pyTDFSDK\classes.py", line 102, in __init__
  File "pyTDFSDK\classes.py", line 125, in get_db_tables
  File "pyTDFSDK\classes.py", line 125, in <dictcomp>
  File "pandas\io\sql.py", line 443, in read_sql_query
  File "pandas\io\sql.py", line 2139, in read_query
  File "pandas\io\sql.py", line 172, in _wrap_result
  File "pandas\core\frame.py", line 2073, in from_records
  File "pandas\core\internals\construction.py", line 807, in to_arrays
  File "pandas\core\internals\construction.py", line 912, in _finalize_columns_and_data
  File "pandas\core\internals\construction.py", line 997, in _convert_object_array
  File "pandas\core\internals\construction.py", line 997, in <listcomp>
  File "pandas\core\internals\construction.py", line 993, in convert
  File "pandas\_libs\lib.pyx", line 2449, in pandas._libs.lib.maybe_convert_objects
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 184. MiB for an array with shape (24083136,) and data type uint64
Process Process-11:
Traceback (most recent call last):
  File "multiprocessing\process.py", line 297, in _bootstrap
  File "multiprocessing\process.py", line 99, in run
  File "source\BrukerTimsTof2mza.py", line 37, in parse_partition
  File "pyTDFSDK\classes.py", line 102, in __init__
  File "pyTDFSDK\classes.py", line 125, in get_db_tables
  File "pyTDFSDK\classes.py", line 125, in <dictcomp>
  File "pandas\io\sql.py", line 443, in read_sql_query
  File "pandas\io\sql.py", line 2139, in read_query
  File "pandas\io\sql.py", line 172, in _wrap_result
  File "pandas\core\frame.py", line 2073, in from_records
  File "pandas\core\internals\construction.py", line 807, in to_arrays
  File "pandas\core\internals\construction.py", line 912, in _finalize_columns_and_data
  File "pandas\core\internals\construction.py", line 997, in _convert_object_array
  File "pandas\core\internals\construction.py", line 997, in <listcomp>
  File "pandas\core\internals\construction.py", line 993, in convert
  File "pandas\_libs\lib.pyx", line 2448, in pandas._libs.lib.maybe_convert_objects
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 184. MiB for an array with shape (24083136,) and data type int64
Process Process-14:
Traceback (most recent call last):
  File "multiprocessing\process.py", line 297, in _bootstrap
  File "multiprocessing\process.py", line 99, in run
  File "source\BrukerTimsTof2mza.py", line 37, in parse_partition
  File "pyTDFSDK\classes.py", line 102, in __init__
  File "pyTDFSDK\classes.py", line 125, in get_db_tables
  File "pyTDFSDK\classes.py", line 125, in <dictcomp>
  File "pandas\io\sql.py", line 443, in read_sql_query
  File "pandas\io\sql.py", line 2139, in read_query
  File "pandas\io\sql.py", line 172, in _wrap_result
  File "pandas\core\frame.py", line 2073, in from_records
  File "pandas\core\internals\construction.py", line 807, in to_arrays
  File "pandas\core\internals\construction.py", line 912, in _finalize_columns_and_data
  File "pandas\core\internals\construction.py", line 997, in _convert_object_array
  File "pandas\core\internals\construction.py", line 997, in <listcomp>
  File "pandas\core\internals\construction.py", line 993, in convert
  File "pandas\_libs\lib.pyx", line 2447, in pandas._libs.lib.maybe_convert_objects
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 367. MiB for an array with shape (24083136,) and data type complex128
...

following machine specs

C:\Windows\System32>systeminfo

Host Name:                 DMED7596
OS Name:                   Microsoft Windows 11 Education
OS Version:                10.0.22635 N/A Build 22635
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Member Workstation
OS Build Type:             Multiprocessor Free
Registered Owner:          N/A
Registered Organization:   N/A
Product ID:                00328-10000-00001-AA719
Original Install Date:     02-Dec-23, 08:57:01 AM
System Boot Time:          27-Feb-25, 10:35:13 AM
System Manufacturer:       Dell Inc.
System Model:              Precision Tower 7910
System Type:               x64-based PC
Processor(s):              2 Processor(s) Installed.
                           [01]: Intel64 Family 6 Model 63 Stepping 2 GenuineIntel ~3401 Mhz
                           [02]: Intel64 Family 6 Model 63 Stepping 2 GenuineIntel ~3401 Mhz
BIOS Version:              Dell Inc. A10, 08-Jan-16
Windows Directory:         C:\WINDOWS
System Directory:          C:\WINDOWS\system32
Boot Device:               \Device\HarddiskVolume8
System Locale:             en-us;English (United States)
Input Locale:              no;Norwegian (Bokmal)
Time Zone:                 (UTC+01:00) Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna
Total Physical Memory:     130,994 MB
Available Physical Memory: 51,816 MB
Virtual Memory: Max Size:  130,994 MB
Virtual Memory: Available: 19,292 MB
Virtual Memory: In Use:    111,702 MB
Page File Location(s):     N/A
Domain:                    win.ntnu.no
Logon Server:              \\WIN-DC2
Hotfix(s):                 4 Hotfix(s) Installed.
                           [01]: KB5037591
                           [02]: KB5031483
                           [03]: KB5032283
                           [04]: KB5032282
Network Card(s):           2 NIC(s) Installed.
                           [01]: Intel(R) I210 Gigabit Network Connection
                                 Connection Name: Ethernet 2
                                 Status:          Media disconnected
                           [02]: Intel(R) Ethernet Connection I217-LM
                                 Connection Name: Ethernet 3
                                 DHCP Enabled:    Yes
                                 DHCP Server:     129.241.0.205
                                 IP address(es)
                                 [01]: 10.20.93.229
                                 [02]: fe80::dbd5:6548:a80f:aa5d
Hyper-V Requirements:      A hypervisor has been detected. Features required for Hyper-V will not be displayed.

C:\Windows\System32>

any way to circumvent this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant