JavaScript heap out of memory errors after upgrading 1.7.1 -> 1.8.0 #3679
Comments
|
Thank you for reporting this issue. We will be triaging your incoming issue as soon as possible. |
msft-github-bot
added
the
Needs: Triage 🔍
|
We have the same problem. Our Setup/Config: What we've tried out so far:
Also we see that the subtask webpack takes more than 1min, and our "rebuild" after a change on running gulp serve takes over 2 Minutes. What is the recommendation for projects with many WebParts, should we move them into several bundles or own projects? Hope this information helps and it can be improved soon |
|
It doesn't appear to be directly related to number of web parts, or size of the bundle as I only have 1 web part. Considering my webpack subtask runtime (~8 secs in our case), I'd say my project is still very much smaller compared to yours - yet same issue is occurring. What would be interesting to know is if this is just issue occurring during development time, so is it safe to deploy the solution. |
|
We have the same issue, but it fails on the webpack step. When downgrading to v5 it bundles fine but we'd like to use the latest. Seems like maybe there's a loop somewhere just eating memory? There are more issues with the fabric for react package in relation to SPFx, but upgrading typescript seems to solve those (ie #3622). Could it be related to #3618 (comment), where pat miller says internally they are building with typescript 2? |
|
I am seeing the same issue. It seems to be related to using office-ui-fabric-react v6. But why v6 and SPFx don't work together...I can't figure that out |
|
Fun fact, if you increase the max memory for node it will bundle/serve eventually. Using this to set the memory to a max of around 4gb, it works for me: For our project it seems to take around 3.5 gb of memory to complete it's tasks. Does make me wonder why it's using to much memory for the new fabric...? |
|
It peaks at around 3.2 GB in my case at the webpack stage, also CPU usage is constant 30%. Have now waited for few minutes it to complete.
And turns out it even crashes with 4000 space_size, but not nearly as often (this is the first time in 2 days). |
|
We are also having a similar problem. The solution has been upgraded from SPFx 1.7.0 to version 1.8.0 however, in our case this error appears when it is performing the tslint subtask. This occurs with gulp build, serve or bundle: By removing a lot of the rules from the tslint.json file the problem goes away however we know this isn't the solution and the underlying problem is elsewhere. I can confirm similarities as well with the memory issues being described by @jpalo |
|
This smells like a common issue due to large projects, such as SPFx+Fabric React v6 (which is now supported with SPFx v1.8). As suggested above, try building with the gulp bundle --max_old_space_size=8192 --ship |
andrewconnell
added
area:tooling
|
Yup we have this issue for one of larger projects as well when upgrading to 1.8 and TypeScript 3.3.4000 Does this have something to do with multiple versions of TypeScript being present in the build setup now? And hence being loaded in memory during bundling? I have noticed the build system still uses different TS versions for different internal packages and and top of that we can use another TS version in our webparts. (The disk size of node_modules has also increased from ~450mb to ~650mb due to the multiple TS versions) |
|
@andrewconnell it does help increasing the memory but it seems to be some sort of leak. If you are running gulp serve and make changes on your code it will build over and over until it reaches the memory limit. Besides the fact that the build time goes up by about 400% every time. So this is definitley not a solution, but can help to at least build the project once. Adding @patmill and @VesaJuvonen since this seems to be an issue for many of us. I am sure they can add the right people to get a fix/solution for that. |
andrewconnell
added
internal:priority
and removed
Needs: Author Feedback
|
I have similar issues (slower builds for each build and eventually a mem leak) in the Teams generator with nodemon/webpack, that we likely think is related to the ts-loader. |
|
We are looking into this internally and will be updating this issue as soon as possible when we have additional things to report. |
|
Same problem here, but at copy-assets: `[12:49:33] Finished subtask 'configure-webpack-external-bundling' after 962 μs <--- Last few GCs ---> [3928:0000022DD5434290] 69925 ms: Mark-sweep 1228.0 (1495.5) -> 1227.9 (1492.5) MB, 277.9 / 0.0 ms allocation failure GC in old space requested <--- JS stacktrace ---> ==== JS stack trace ========================================= Security context: 0000011ADFB258B9 FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory |
|
Same problem as well during webpack step: What is interesting for me is I went through the upgrade steps provided by Office 365 CLI and began serving the project successfully yesterday evening. This morning, I removed package-lock.json and node_modules and re-ran npm install. After performing those steps is when this error began happening for me. Project contains 12 web parts and 2 extensions. Adding --max-old-space-size=4000 to serve command allows me to debug |
|
this |
|
Looking into this a bunch, and have the same results. Namely, with TS3.x I get this a lot more. I've found that bypassing some tslint errors can make it go away as well, but that seems like a bad trade-off. The --max_old_space_size is the best solution. We're going to work on a way to make that easier to use (the latest sp-starter-kit package.json file has the commands as short-cuts.). Unfortunately we can't update the gulp file, because it is too late at that point. So, nothing really to add here, other than to say "we're working on making this less of a problem". |
|
Thanks @patmill, so do you mean this is not some sort of memory leak as we previously thought and just a result of the toolchain needing more memory now for large projects? |
|
There could still be some sort of tooling leak, or possibly when you live updates the node process lives on longer w/ more memory consumption. I was finding that build sp-starter-kit would flat out fail from the get-go. Will continue to dig in further, but if the community has findings of their own, that would certainly help. |
|
@patmill There is a way better workaround. Using --max_old_space_size does work, but the time it takes for webpack in our larger solution (30+ webparts/extensions) is over 4 minutes compared to under 1 minute with spfx 1.7 and office-ui-fabric-react 5.x The issue is clearly related to office-ui-fabric-react. So what I did is adding office-ui-fabric-react as external dependency in config.json and everything works fine again. BUT: This is not a supported as we all know. So i created a custom bundle of office-ui-fabric-react as AMD module with an own namespace and a custom npm alias and so on to prevent collision with the things Microsoft is using internaly and everything is faster and better than ever before ;) If you would provide an official way to support office-ui-fabric-react as an external dependency of our project, all issues would be solved. In our case the webpack task is now down to 12 seconds....compared to over 4 minutes, that is huge! :) |
|
@OliverZeiser said:
Haven't they already done that with the library component type (granted its dev preview now)? https://docs.microsoft.com/en-us/sharepoint/dev/spfx/library-component-overview |
andrewconnell
added
the
status:working-on-it
|
@andrewconnell it is not that easy to convert the office-ui-fabric-react package to an spfx library bundle. I have tried that as well but ran into too many issues. The office-ui-fabric-react has some very special build processes that kick in during a build and I did not have the time to dig in any deeper. Also I think it needs webpack 4 where SPFx is still using webpack 3? But if someone would figure out how to make an spfx library package out of the OUIFR repo it could indeed work. And would be a very nice solution. |
|
@andrewconnell adding to links regarding the support statements: |
|
Can confirm this is fixed for us now! Thanks 👍 |
|
thx @vman for the confirmation also from your side. We will keep on optimizing the developer experience even further with the upcoming 1.9 and following versions. |
|
@VesaJuvonen I can NOT confirm that this is working. In our solution i still get the error after upgrading to 1.8.2 and node 10. |
|
I can confirm that this issue has been resolved for us also. Thanks guys and great work 👍 |
|
I am also still getting the out of memory error after upgrading to 1.8.2. I’ve tried on node 8 and node 10 with same results. For me, I only get the error in a large project, 16 web parts and 2 extensions. If a new issue should be opened, I can do so and provide screens/package.json if needed. |
|
After further testing, by moving OUIFR to external dependency as @OliverZeiser mentioned, the memory error goes away. Also of note, this has reduced my build time from 2.5-3 mins down to 39 seconds. |
|
Can you give the versions of the build and tooling you are using in your package.json? ie, which rush-stack-compiler, as well as which version of office-ui-fabric-react? |
|
Sure thing, see below and let me know if you need anything further.
I'm using 3.2 of compiler:
|
|
You don't want multiple @microsoft/rush-stack-compiler-#-#& package installed.. you only want one. In this case, you only want 3.2. Try a quick fix of (1) uninstalling the other two, (2) delete node_modules & (3) reinstall all packages with |
|
Thanks @andrewconnell for the suggestion. I followed the steps, removing 2.7 and 3.3, deleting node_modules, then reinstalling everything. I still get the error on both node 8 and 10. node 8:
node 10
|
|
Let's try a few things: If that doesn't work, try using "@microsoft/rush-stack-compiler-2.9": "0.7.7" instead (and update your tsconfig.json file as well) after this, can you run |
|
(weird, my response appears to have not posted). You've got a lot of extra stuff in your package.json file. Can you remove: then run and respond here with the contents of You'll then want to clean / rebuild. If that doesn't work, can you switch your rush-stack-compiler to version [email protected] and go through those steps again? You'll need to update your tsconfig.json file to point to the -2.9 path. |
|
Thanks @patmill my responses and posts are all out of order for some reason. Not sure if you saw my results after removing the additional rush stack compiler entries, but I still get the error. I will try your suggestions as well and report back. |
|
Wow - the out-of-order things are making my brain hurt. @c-eiser13 - can you give the results of |
|
Ha, agree @patmill , it is very hard to follow.
|
|
Very strange... GH is posting some comments "in the future". Note the 4 at the bottom of the issue say "X hours from now" where everything else is "X hours ago". That looks buggy... |
|
@c-eiser13 - if you are up for it, can you share your project out? If you want, you can send it to me at my user name here @microsoft.com I used the sp-starter-kit as my stress test for this, and it has pnp, fabric, 17 webparts, 7 extensions, etc. Hopefully we can get a solution for you while we work on the next version of the tools. |
|
Thanks @patmill for the sake of thoroughness, I will complete the steps you mentioned in testing. I only tested removing the 2.7 and 3.3 version of the compiler, I have not tested the other steps you provided. Once I am able to, I will report back and possibly send over to you. To be honest, I can handle adding the max-old-space for the workaround, more importantly, I am hoping a future release will enhance the build times of large projects. Currently while developing, every save I have to wait 1.5-2.5 minutes to see the result. Externalizing OUIFR brings me back to around 30 seconds, so hopefully there will be a way to put this into a library component or gain more control over which components are processed while serving. Thanks again for all your help and responses, and once I'm able to test, I will post back. |
|
Thanks. I'm just wanting to sort out where the issues are for other people as well (or just admit that we need the max-old-space and pivot the tooling to accept that). One last question, around the externalizing of OUIFR. What does that actually look like at runtime? Do you load the entire OUIFR js package? I ask because you might be trading off 2 minutes of build for 800+kb of minimized js loaded onto the page. That's an unhealthy trade off. Maybe externalizing during the inner loop and bundling during the final build is a better approach? At any rate, keep us posted. |
|
@patmill i just added OUIFR to the externals section of config.json and was able to gulp serve successfully in 1/4 the time it had been taking. Once I saw this reply, I tried again and actually went to the site, but all web parts were errored out. I believe it was @OliverZeiser that had a solution where he created his own external bundle of OUIFR, for me, it just allowed me to serve with no error and much quicker than previously. I followed your previous suggestions, removing the generator and typescript from package.json, deleted node_modules, ran npm install, then gulp clean and here are my results.
Perhaps I should have been more specific, in all of my previous testing, the error was always received when running gulp serve (and still is), i was not trying build or bundle. I downgraded compiler to 2.9, and the results were the same as above, error in serve and success in build and bundle. Thanks again for your help and feedback, let me know if there is anything else you would like me to try. I will discuss with my team about sending the solution over to you if that may help find the root cause and let you know. |
I can confirm that this is the case. Changing the memory limit only fixes the first one or two builds triggered by gulp serve. Adding our project configuration as a datapoint (9 web parts total): npm list webpack
package.json
@OliverZeiser I'm a SharePoint newbie - what are the negative implications of adding OUIFR as an external dependency? |
|
OUIFR is not intended to be consumed this way. As such not all components are working as the exports in OUIFR are not 100% correct afaik. |
|
For what it's worth, I haven't seen the issue after migrating to 1.8.2, ceteris paribus. |
|
There's one thing I don't understand. I'm trying the --max_old_space_size workaround on a machine with 32GB of RAM. Watching my resource monitor while running gulp serve shows the node process consuming only ~1.5GB of RAM when it crashes. It doesn't seem like the flag is having any effect. |
|
gulp serve --max_old_space_size=8192 --nobrowser worked for me |
|
I am still having this issue after migrating to 1.9.1. |
@Rafaelki Unfortunately, I can't confirm this, have adapted my project and installed all dependencies via pnpm. Getting a heap out of memory error at Just the Solution mentioned by @OliverZeiser to add OUIFR as externals seems work. |
|
Thank you all who have posted tips. I encountered this issue after upgrading my SFPx project from 1.8.1 to 1.9.1 using the Office 365 CLI project upgrade command (maybe something is missing in the upgrade report?). To fix the issue I changed the rush-stack-compiler from 2.9 to 3.3 and upated my tsconfig.json to reference the rush-stack-compiler change. Then re-insalled packaged and now "gulp sever --nobrowser" works. |
|
Issues that have been closed & had no follow-up activity for at least 7 days are automatically locked. Please refer to our wiki for more details, including how to remediate this action if you feel this was done prematurely or in error: Issue List: Our approach to locked issues |
Uh oh!
There was an error while loading. Please reload this page.
Category
Expected or Desired Behavior
No memory errors when building and working with the code in SP Online workbench.
Observed Behavior
Upgraded from 1.7.1 to 1.8.0 and started seeing this in my project. Occurs every 3-4th time I modify file & save, which triggers build. As you can see, when the issue occurs on the copy-assets step, webpack step takes long (1 min in this case, also had it occur just now with 26 secs runtime). When it doesn't fail, webpack step only takes 6-11 secs.
Package.json
tslint.json
tsconfig.json
Steps to Reproduce
Project contains 1 React-based web part.
I understand this is a bit vague, apologies, but I don't have much idea what to try, or what kind of addition information you would really need to be able to comment on the issue.
The text was updated successfully, but these errors were encountered: