Memory access out of bounds как исправить

Memory access out of bounds как исправить

Reproduction steps:
1. Open the attached «1403906.zip» project
2. Build and Run the project in the Safari browser
3. Observe the game in the Safari browser

Expected result: No error messages appear
Actual result: «RuntimeError: Out of bounds memory access» error message appears

Reproducible with: 2021.2.14f1, 2022.1.0b10, 2022.2.0a7
Could not test with: 2019.4.36f1, 2020.3.30f1 (Compilation errors after downgrade)

Not reproducible with: Chrome and Firefox browsers

First few lines of the error message:
An error occurred running the Unity content on this page. See your browser JavaScript console for more info. The error was:
RuntimeError: Out of bounds memory access (evaluating ‘dynCall_v.call(null,func)’)
.wasm-function[ParticleSystem::UpdateModulesPreSimulationIncremental(ParticleSystemUpdateData const&, ParticleSystemParticles&, unsigned long, unsigned long, math::_float4 const&, bool)]@[wasm code]
.wasm-function[ParticleSystem::Update1b(ParticleSystemUpdateData&, int)]@[wasm code]
.wasm-function[ParticleSystem::UpdateFunction(ParticleSystemUpdateData*)]@[wasm code]
.wasm-function[ParticleSystem::BeginUpdate(ParticleSystem**, unsigned long, float, float)]@[wasm code]
.wasm-function[ParticleSystem::BeginUpdateAll()]@[wasm code]
.wasm-function[ParticleSystem::InitializeClass()::PreLateUpdateParticleSystemBeginUpdateAllRegistrator::Forward()]@[wasm code]
.wasm-function[ExecutePlayerLoop(NativePlayerLoopSystem*)]@[wasm code]

Memory access out of bounds как исправить

Reproduction steps:
1. Open the user’s attached project
2. Set ILC2PP Code Generation to Faster (smaller) builds
3. Build And Run the Project

Expected result: Build runs
Actual result: Build shows «RuntimeError: memory access out of bounds»

Reproducible with: 2022.1.0a12, 2022.1.0a13
Not reproducible with: 2020.3.25f1, 2021.2.0f1, 2022.1.0a11, 2022.1.0a14, 2022.1.0b2

Could not test with: 2019.4.34f1 (error CS0305: Using the generic type ‘Progress’ requires 1 type arguments)

Note: the error is inconsistent, on some versions you need to set ILC2PP Code Generation to Faster (smaller) builds, on some not

RuntimeError : Memory Access Out of Bounds — WebGL

I’ve expected to either get the solution or get aware of what is causing this error.

• unity-game-engine
• runtime-error
• unity-webgl

7,720 2 2 gold badges 25 25 silver badges 35 35 bronze badges
asked Jul 14 at 7:28

Sounds like your app is exceeding one of the WebGL specific memory limits — blog.unity.com/engine-platform/…

Jul 14 at 12:40
Related questions
No ‘Access-Control-Allow-Origin’ in webGL Unity Multiplayer
Memory Access Out of Bounds — WebGL
Blazor WASM RuntimeError memory out of bounds

0

Know someone who can answer? Share a link to this question via email, Twitter, or Facebook.

• unity-game-engine
• runtime-error
• unity-webgl

The Overflow Blog

sponsored post

Linked

Hot Network Questions

Subscribe to RSS

Question feed

To subscribe to this RSS feed, copy and paste this URL into your RSS reader.

Site design / logo © 2023 Stack Exchange Inc; user contributions licensed under CC BY-SA . rev 2023.11.15.1019

By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.

Saved searches

Use saved searches to filter your results more quickly

Cancel Create saved search

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

pyodide / pyodide Public

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory access out of bounds in Chrome #1384

dmondev opened this issue Mar 28, 2021 · 67 comments · Fixed by #1449

Memory access out of bounds in Chrome #1384

dmondev opened this issue Mar 28, 2021 · 67 comments · Fixed by #1449
Blocker bug Something isn’t working

Comments

Contributor
dmondev commented Mar 28, 2021

I’m getting the following intermittent error running the develop version in Chrome, running from a web worker.

Chrome Version 89.0.4389.90 (Official Build) (64-bit)
Windows 10 1909 build 18363.1379

pyodide.js:406 The cause of the fatal error was:
RuntimeError: memory access out of bounds
at PyArray_IntpFromIndexSequence (:wasm-function[991]:0x737a1)
at PyArray_IntpConverter (:wasm-function[990]:0x73702)
at byn$fpcast-emu$PyArray_IntpConverter (:wasm-function[3415]:0x15bfeb)

The text was updated successfully, but these errors were encountered:

hoodmane commented Mar 28, 2021

I’ve seen this a couple of times too, but not enough times to have any idea of a pattern for when it happens. If anyone can figure out a way to reproduce it, that would be super helpful.

hoodmane commented Mar 28, 2021

Also it would be useful to know whether it is browser dependent, so if anyone has seen a similar crash in firefox that’d be interesting info.

Contributor Author
dmondev commented Mar 28, 2021

Weirdly enough it happens less when I have dev tools open. I’ll try to reproduce with a minimal setup/MRE.

hoodmane commented Mar 28, 2021
Contributor
leopsidom commented Mar 28, 2021

+1 on open dev tool makes this happen less frequently. Another pattern I observe is that this seems to happen more frequently on Mac (chrome) than Windows (chrome).

Contributor Author
dmondev commented Mar 29, 2021 •

I can now say that this happens constantly when Chrome DEV tools is NOT open.

The issue doesn’t happen in Firefox.

I’m able to reproduce with the following codepen: https://codepen.io/dmondev/pen/eYgdBmm

This is as close as possible to how I’m using pyodide (in a worker, with custom namespaces).

Also, the error goes away if I remove the imports

import pandas as pd import numpy as np

hoodmane commented Mar 29, 2021

@dmondev I can confirm the reproduction on my computer, thanks!

hoodmane commented Mar 29, 2021

Note that the problem goes away here if you import numpy before pandas. Have you tried that in your actual use case? Presumably the error just appears further down the road?

Contributor Author
dmondev commented Mar 29, 2021

Done. updated the codepen with output to the document itself.

Yes, the error happens further down the road . so dependency order issues ?

hoodmane commented Mar 29, 2021

It’s confusing because pandas depends on numpy so by the time pandas is imported, numpy must have been imported too.

Does the error always happen inside of PyArray_IntpFromIndexSequence ?

Contributor Author
dmondev commented Mar 29, 2021 •

It’s confusing because pandas depends on numpy so by the time pandas is imported, numpy must have been imported too.

Does the error always happen inside of PyArray_IntpFromIndexSequence ?

It appears so.
Also, I cleaned up everything that is not necessary to reproduce the issue in the codepen (https://codepen.io/dmondev/pen/eYgdBmm)

Removed pandas as well. Issue seems to be with loading numpy.

Contributor Author
dmondev commented Mar 29, 2021

Additional note: Not happening in v0.17.0a1

hoodmane commented Mar 29, 2021

This probably originated when #887 was merged (since the bug only really has to do with numpy, loading pandas just helps to trigger it). @rth

hoodmane commented Mar 29, 2021

If this hypothesis is correct, the bug shouldn’t appear on commit fdfc56f and should appear starting with commit d175636.

rth commented Mar 29, 2021 •

This probably originated when #887 was merged

I don’t think that’s it, since #887 is not included in 0.17.0a2.

It seems like the dev version consumes more memory than the previous version ?

@leopsidom 0.17.0a1 (or the dev version) should consume less. Doing memory snapshots in Firefox 87,

• 0.16.1: empty REPL -> 55MB, additionally load numpy -> 115 MB (total)
• dev version: empty REPL -> 52 MB , additionally load numpy -> 85 MB (total)

In what code did you see a memory increase?

rth commented Mar 29, 2021 •

Additional note: Not happening in v0.17.0a1

We might be able to git bisect it between 0.17.0a1 and 0.17.0a2 using CircleCI artifacts of commits on master, though those are only kept for a month I think (#648). Personally I am not able to reproduce with the provided codepen in Chrome 88 on Linux (with the dev tools closed).

Contributor
jmsmdy commented Mar 29, 2021

Can confirm both codepens also crash consistently with dev tools closed on MacOS.

MacOS: Version 10.15.7 (19H524)
Chrome: Version 89.0.4389.90 (Official Build) (x86_64)

Contributor
leopsidom commented Mar 29, 2021

@rth I don’t think my assumption was correct now. I had the belief based on page crash info given by chrome, which I think is too general to identify the issue. But I double checked the memory on chrome. It was somewhere around 50 ~ 60MB IIRC after pandas is loaded, which should be far below the per tab memory limit (I believe it’s at least GB level based on my research). So I guess the issue is due to something else.

hoodmane commented Mar 29, 2021

The memory limit might be lower in a webworker, though I have seen this a couple of times on the main thread.

Also, I am no longer getting either codepen to reproduce the issue, I only ever got the original version to show the error. So it seems to be pretty inconsistent.

Contributor Author
dmondev commented Mar 29, 2021 •

I pasted the codepen twice, but it’s just one. As I made a few changes to cleanup, just want to make sure we are all seeing the same thing:
https://codepen.io/dmondev/pen/eYgdBmm

Contributor
jmsmdy commented Mar 29, 2021 •

@rth I don’t think my assumption was correct now. I had the belief based on page crash info given by chrome, which I think is too general to identify the issue. But I double checked the memory on chrome. It was somewhere around 50 ~ 60MB IIRC after pandas is loaded, which should be far below the per tab memory limit (I believe it’s at least GB level based on my research). So I guess the issue is due to something else.

I’m seeing higher than that in Chrome Task Manager:

Also, Chrome sometimes groups multiple tabs per process, which could possible push the memory up.

Googling a bit about this issue, one common cause of «memory access out of bounds» is apparently an object being garbage-collected before use. «Intp» stands for «integer pointer», which I’m guessing numpy uses for things like allowing a slice of an array to point to the same memory location as the original array. Perhaps there is some over-eager garbage collection happening?

Since it happens when numpy is imported, there may be some platform tests that happen on load (e.g., «try creating such and such arrays, and depending on which ones fail, set the following parameters»).

hoodmane commented Mar 29, 2021 •
hoodmane commented Mar 29, 2021

I should update the fatal error handling so that it prints the current Python stack frame if possible.

hoodmane commented Mar 29, 2021 •

Okay using _Py_DumpTraceback I managed to extract the Python traceback:

File "/lib/python3.8/site-packages/numpy/core/numerictypes.py", line 853 in File "/lib/python3.8/site-packages/numpy/core/_internal.py", line 18 in File "/lib/python3.8/site-packages/numpy/core/__init__.py", line 35 in File "/lib/python3.8/site-packages/numpy/lib/type_check.py", line 11 in File "/lib/python3.8/site-packages/numpy/lib/__init__.py", line 8 in File "/lib/python3.8/site-packages/numpy/add_newdocs.py", line 13 in File "/lib/python3.8/site-packages/numpy/__init__.py", line 142 in File "", line 1 in File "/lib/python3.8/site-packages/pyodide/_base.py", line 270 in run_async File "/lib/python3.8/site-packages/pyodide/_base.py", line 419 in eval_code_async File "/lib/python3.8/asyncio/tasks.py", line 280 in __step File "/lib/python3.8/asyncio/events.py", line 81 in _run 

hoodmane commented Mar 29, 2021 •
Contributor
jmsmdy commented Mar 29, 2021 •

 else: _typestr[key] = empty((1,), key).dtype.str[1:] 

we see a call to «empty», which is imported from numpy.core.multiarray (compiled from C).
Taking a look inside multiarraymodule.c, in the definition of ‘array_empty’ (exported as python ’empty’): https://github.com/numpy/numpy/blob/v1.15.4/numpy/core/src/multiarray/multiarraymodule.c#L1887

 if (!PyArg_ParseTupleAndKeywords(args, kwds, "O&|O&O&:empty", kwlist, PyArray_IntpConverter, &shape, PyArray_DescrConverter, &typecode, PyArray_OrderConverter, &order))

We see a use of PyArray_IntpConverter (in this case, it looks like it is called by this function PyArg_ParseTupleAndKeywords).

 seq->len = len; nd = PyArray_IntpFromIndexSequence(obj, (npy_intp *)seq->ptr, len); if (nd == -1 || nd != len)  

diff --git a/packages/numpy/meta.yaml b/packages/numpy/meta.yaml index 2bd617e..83d4540 100644 --- a/packages/numpy/meta.yaml +++ b/packages/numpy/meta.yaml @@ -22,7 +22,8 @@ source: build: skip_host: False - cflags: -include math.h -I../../config -Werror=implicit-function-declaration -Werror=mismatched-parameter-types -Werror=mismatched-return-types + cflags: -O0 -include math.h -I../../config -Werror=implicit-function-declaration -Werror=mismatched-parameter-types -Werror=mismatched-return-types + ldflags: -O0 -s ASSERTIONS=1 post: | # copy the correct numpy config into the build artifacts. Otherwise scipy will try to build with the config # from the build computer and bad things will happen

export EXTRA_CFLAGS="-g4" export EXTRA_LD_FLAGS="-g4 --source-map-base=localhost:8000" export PYODIDE_PACKAGES="numpy" make

import socket import socketserver from http.server import SimpleHTTPRequestHandler import pathlib alternative_bases=["cpython/build/Python-3.8.2/","src/", "build/"] def fixup_url(path): if pathlib.Path("." + path).exists(): return path for base in alternative_bases: q = pathlib.Path(base + path) if q.exists(): return str(q) dir = list( pathlib.Path("packages/numpy/build/numpy-1.17.5/").glob("**" + path), ) if dir: return str(dir[0]) print("not found:", path) return path class MyTCPServer(socketserver.TCPServer): def server_bind(self): """Use socket.SO_REUSEADDR to allow faster restart""" self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) self.socket.bind(self.server_address) class Handler(SimpleHTTPRequestHandler): def end_headers(self): # Enable Cross-Origin Resource Sharing (CORS) self.send_header('Access-Control-Allow-Origin', '*') super().end_headers() def do_GET(self): self.path = fixup_url(self.path) super().do_GET() if __name__ == '__main__': port = 8000 with MyTCPServer(("", port), Handler) as httpd: print("Serving at: http://127.0.0.1:<>".format(port)) httpd.serve_forever()

vals[i] = PyArray_PyIntAsIntp(op); Py_DECREF(op); if(vals[i] == -1) < // error happened here

(func $PyArray_IntpFromIndexSequence (;1429;) (export "PyArray_IntpFromIndexSequence") (param $var0 i32) ;; seq (param $var1 i32) ;; vals (param $var2 i32) ;; maxvals, reused for op and err (result i32) (local $var3 i32) ;; i / loop variable (local $var4 i32) ;; return value (so sometimes nd) (local $var5 i32) ;; nd (local $var6 i32) ;; min(nd, maxvals) (local $var7 i32) ;; &vals[i] (local $var8 i32) ;; used for op->obj_refcnt (in code from Py_DECREF) block $label0 ;; wraps the whole body, we use jump to $label0 to return $var4 ;; line 931 local.get $var0 call $PySequence_Size local.tee $var5 ;; nd 5 = PySequence_Size(seq 0) ;; line 932 i32.const -1 i32.ne if ;; no error occurred ;; line 952, computing min(nd, maxvals) local.get $var5 local.get $var2 local.get $var2 local.get $var5 i32.gt_s select local.tee $var6 ;; set var6 = min(nd, maxvals) i32.const 0 i32.gt_s if ;; min(nd, maxvals) > 0, (otherwise loop skipped entirely) i32.const -1 local.set $var4 ;; return value is -1 loop $label2 ;; line 953 local.get $var0 ;; seq local.get $var3 ;; i (loop variable) call $PySequence_GetItem ;; GetItem(seq, i) local.tee $var2 ;; op = GetItem(seq, i) (overwrite maxvals) ;; line 954 i32.eqz ;; if(op == NULL) ;; line 955 br_if $label0 ;; return -1 (in $var4) ;; line 958, start with LHS local.get $var1 ;; vals local.get $var3 ;; i i32.const 2 i32.shl ;; i i32.add ;; &vals[i] == vals + 4*i local.tee $var7 ;; v7 = &vals[i] ;; inlined PyArray_PyIntAsIntp here local.get $var2 ;; op ;; fetch static string "an integer is required" global.get $__memory_base i32.const 9084 i32.add call $PyArray_PyIntAsIntp_ErrMsg ;; PyArray_PyIntAsIntp_ErrMsg(o, "an integer is required") i32.store ;; vals[i] = PyArray_PyIntAsIntp(op) ;; line 959, Py_DECREF macro local.get $var2 local.get $var2 i32.load ;; op->ob_refcnt i32.const 1 i32.sub ;;  local.tee $var8 i32.store ;; store back op->ob_refcnt -= 1 local.get $var8 i32.eqz if ;; if new refcount is 0, dealloc op local.get $var2 call $_Py_Dealloc end ;; Py_DECREF is done now block $label1 ;; line 960 ----------------------------------------------------------------------------------- local.get $var7 ;; &vals[i] i32.load ;; vals[i] ;; i32.const -1 i32.ne ;; vals[i] != -1 br_if $label1 ;; no error, continue loop ;; line 961 ------------------------------------------------------------------------------------ call $PyErr_Occurred local.tee $var2 ;; store err into $var2 i32.eqz br_if $label1 ;; no error, continue loop ;; Now we know there was an error, replace overflow error message local.get $var2 global.get $PyExc_OverflowError i32.load call $PyErr_GivenExceptionMatches i32.eqz br_if $label0 ;; Not an overflow error, return -1 ($var4) ;; An overflow error, replace error message ;; lines 964 / 965 global.get $PyExc_ValueError i32.load ;; Look up static string "Maximum allowed dimension exceeded" global.get $__memory_base local.tee $var2 i32.const 9107 i32.add call $PyErr_SetString i32.const -1 return end $label1 ;; Now we need to decide whether to loop again ;; line 952 again. increment i then test whether we are done local.get $var3 ;; i / loop variable i32.const 1 i32.add ;; i + 1 local.tee $var3 ;; store back local.get $var6 ;; MIN(nd, maxvals) i32.ne ;; i != MIN(nd, maxvals) br_if $label2 ;; loop unless we're done (?) end $label2 end local.get $var5 local.set $var4 ;; move nd from $var5 into $var4 (now that we succeeded) br $label0 ;; return nd (in $var4) end ;; 933 (an error occurred in PySequenceLength(seq)) call $PyErr_Occurred if ;; line 934 call $PyErr_Clear end ;; line 937 ;; inlined PyArray_PyIntAsIntp again local.get $var1 ;; vals (used by i32.store for assign to vals[0]) ;; fetch static string "an integer is required" local.get $var0 global.get $__memory_base i32.const 9084 i32.add call $PyArray_PyIntAsIntp_ErrMsg ;; PyArray_PyIntAsIntp_ErrMsg(o, "an integer is required") local.tee $var2 ;; override maxvals with result i32.store ;; vals[0] = PyArray_PyIntAsIntp_ErrMsg(o, "an integer is required") i32.const 1 ;; line 949 local.set $var4 ;; nd = 1 ;; line 938 local.get $var2 ;; vals[0] i32.const -1 i32.ne ;; vals[0] != -1 br_if $label0 ;; if vals[0] != -1 return nd (in $var4) ;; line 939 call $PyErr_Occurred local.tee $var2 ;; use $var2 as err ;; line 940/941 i32.eqz ;; err == 0 br_if $label0 ;; if(err == 0) return nd (in $var4) ;; Now there was an error. i32.const -1 local.set $var4 ;; there was an error, we're going to return -1 local.get $var2 global.get $PyExc_OverflowError i32.load call $PyErr_GivenExceptionMatches ;; PyErr_GivenExceptionMatches(err, PyExc_OverflowError) i32.eqz br_if $label0 ;; not an overflow error, return -1 (in $var4) ;; An overflow error, replace error message ;; lines 942/943 global.get $PyExc_ValueError i32.load ;; get static string "Maximum allowed dimension exceeded" global.get $__memory_base local.tee $var2 i32.const 9107 i32.add call $PyErr_SetString i32.const -1 return end $label0 local.get $var4 )