RAM/memory allocation with LCOW/windows containers? #1357

rlpowell · 2017-11-24T20:42:45Z

Expected behavior

I expected LCOW mode to give my containers access to all of my Windows' box's RAM.

Actual behavior

The RAM appears to be capped to 1G, per /proc/meminfo

Also, there is no longer anything in the Docker Settings to change it.

Information

B20175AC-8170-4955-8409-C1846A7F82E7/2017-11-24_12-20-54

C:\Users\rlpowell>systeminfo | findstr em
System Boot Time:          2017-11-24, 11:51:51
System Manufacturer:       LENOVO
System Model:              20AQCTO1WW
System Type:               x64-based PC
System Directory:          C:\WINDOWS\system32
System Locale:             en-us;English (United States)
Total Physical Memory:     7,901 MB
Available Physical Memory: 4,486 MB
Virtual Memory: Max Size:  17,071 MB
Virtual Memory: Available: 7,411 MB
Virtual Memory: In Use:    9,660 MB
Hyper-V Requirements:      A hypervisor has been detected. Features required for Hyper-V will not be displayed.

C:\Users\rlpowell>docker run -it rlpowell/shell head /proc/meminfo
MemTotal:         995516 kB
MemFree:          828124 kB
MemAvailable:     763408 kB
Buffers:            2668 kB
Cached:            20488 kB
SwapCached:            0 kB
Active:            15280 kB
Inactive:          10580 kB
Active(anon):      12572 kB
Inactive(anon):    10572 kB

C:\Users\rlpowell>docker run -it rlpowell/shell free -g
              total        used        free      shared  buff/cache   available
Mem:              0           0           0           0           0           0
Swap:             0           0           0

C:\Users\rlpowell>docker run -it rlpowell/shell free -m
              total        used        free      shared  buff/cache   available
Mem:            972         113         808          20          50         745
Swap:             0           0           0

Steps to reproduce the behavior

Run any linux contain in Windows Container mode.

The text was updated successfully, but these errors were encountered:

rlpowell · 2017-11-25T05:35:10Z

This appears, at first glance, to be equivalent to microsoft/opengcs#145 , which is linked from moby/moby#33850 .

(Which is unfortunate, as several of my compilation steps require 2G just to show up, so I can't test LCOW at all ;_; )

liquidboy · 2018-01-30T10:33:48Z

I encountered this limitation when trying to use Node in a LCOW container where node was trying to go above 2GB and hitting an "out of memory exception". I was using Node in a LCOW as a build agent. Please let us increase this hard limit above 2GB..

This is the snippet of the "sh" script we run that shows us calling Node saying its ok to go up to 3GB (-max-old-space-size) . This is common in build agents building complex repositories.. [the example i use is trying to build the ANGULAR repository..

NGC="node --max-old-space-size=3000 pwd/dist/tools/@angular/compiler-cli/src/main"

Below is the Node error i got showing the "process out of memory" in Node

<--- Last few GCs --->

[818:0x25e0a70] 21353 ms: Scavenge 685.4 (727.1) -> 673.7 (729.1) MB, 10.5 / 0.0 ms allocation failure
[818:0x25e0a70] 21411 ms: Scavenge 688.2 (730.1) -> 676.6 (732.1) MB, 9.9 / 0.0 ms allocation failure
[818:0x25e0a70] 21515 ms: Scavenge 690.9 (733.1) -> 679.2 (735.1) MB, 10.1 / 0.0 ms allocation failure
[818:0x25e0a70] 21580 ms: Scavenge 693.4 (735.6) -> 682.6 (738.1) MB, 11.7 / 0.0 ms allocation failure

<--- JS stacktrace --->
Cannot get stack trace in GC.
FATAL ERROR: NewSpace::Rebalance Allocation failed - process out of memory
1: node::Abort() [node]
2: 0x121a2cc [node]
3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
5: 0xaea93b [node]
6: v8::internal::MarkCompactCollector::Evacuate() [node]
7: v8::internal::MarkCompactCollector::CollectGarbage() [node]
8: v8::internal::Heap::MarkCompact() [node]
9: v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
10: v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
11: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node]
12: v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
13: 0x10eea048463d
./build.sh: line 403: 818 Aborted $TSC -p ${TSCONFIG}

fizxmike · 2018-04-24T19:14:39Z

Any update on this? I'm launching a java app in a container and it freaks out that it "sees" only ~900 MB available and dies (of course things work find on linux docker host).

The closest answer I got was in this cryptic post mentioning JOB_OBJECT_LIMIT_JOB_MEMORY, but I don't know how to implement, my Windows Foo is very weak.

FYI, I'm running windows server 1709 and docker (17.10.0-ee-preview-3)

docker-robott · 2018-07-31T01:41:05Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

nerumo · 2018-07-31T07:09:48Z

/remove-lifecycle stale

bloodybeet · 2018-08-31T15:30:35Z

Could someone please provide an estimate for when we can run LCOW with +2GB RAM? We would very much like to use LCOW mostly to be able to bind to localhost whilst still using linux containers (or has anyone found out how to do that with the MobyLinux VM?)

icenine457 · 2018-09-14T13:19:16Z

Also looking for this issue to be fixed, as I'm containerizing a ASP.Net application using SQL Server, which requires 2 GB RAM to run.

greyphoenix · 2018-09-30T01:56:52Z

+1 for this.

pccruiser · 2018-10-06T14:38:35Z

+4096 for this!

Same situation. Want to use SQL Linux in a container while also testing asp.net application that requires windows. No reason this can't be done other than an arbitrary memory limit! I have 32GB of RAM so that is very frustrating!!

jimpriest · 2018-10-07T19:45:16Z

Just ran into this myself. Not enough memory.

nayala0791 · 2018-11-20T18:41:21Z

/remove-lifecycle stale

KwalityKoder · 2018-11-21T18:22:46Z

This is a massive blocker for us. We have a Linux container running WildFly that needs (and gets 3Gb) when Docker for Windows is running in Linux Container mode.
Switching Docker for Windows to Windows Container mode, I can see (using cat /proc/meminfo) that the same container is only given less than 1Gb of memory, and WildFly consequently dies.

bloodybeet · 2018-11-21T18:29:59Z

I almost can't believe how much of a blocker this is for real-world usage for so long for so many of us. As if it's being witheld on purpose, to fix some other issues first? Op wo 21 nov. 2018 om 19:22 schreef Mark Hemingway <notifications@github.com

…

: This is a massive blocker for us. We have a Linux container running WildFly that needs (and gets 3Gb) when Docker for Windows is running in Linux Container mode. Switching Docker for Windows to Windows Container mode, I can see (using cat /proc/meminfo) that the same container is only given less than 1Gb of memory, and WildFly consequently dies. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1357 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALjT3IMR7pVtKYwOXuppnzpvssZAPbllks5uxZn7gaJpZM4QqLGT> .

KwalityKoder · 2018-11-22T12:29:06Z

Can someone from the docker development team, please advise:
a) When this issue is scheduled to be fixed?
b) That there is absolutely no workaround?

SteveTownsendBway · 2018-11-22T13:31:06Z

My experience is that this can be worked around by building your own dockerd using the moby/moby master branch merged (locally) into the pull here moby/moby#37296 that fixes this issue. The merge to master is pending but I could not wait. I am now able to docker run --memory 6144m for example.
In my case this was needed to get Linux SQL Server up and running: needs 2GB, but without fix, hard limit is 1GB.

KwalityKoder · 2018-11-22T14:07:07Z

@SteveTownsendBway We plan to use docker-compose to run our Linux containers via LCOW.
Will moby/moby#37296 fix this issue for docker-compose too?
Will the docker-compose mem_reservation: flag suddenly start working again? Or will a new PR be required for docker-compose?

DecisionSystems · 2018-11-22T14:49:47Z

Problem for us as well. Trying to run .Net Framework container alongside Linux container on Windows 10 Build 17134 and latest Docker with LCOW enabled. Let a guy experiment please!

SteveTownsendBway · 2018-11-22T15:12:41Z

I don’t know Mark. Maybe the implementor or code reviewer can comment. From: Mark Hemingway [mailto:notifications@github.com] Sent: Thursday, November 22, 2018 9:07 AM To: docker/for-win <for-win@noreply.github.com> Cc: Steve Townsend <stownsend@broadwaytechnology.com>; Mention <mention@noreply.github.com> Subject: Re: [docker/for-win] RAM/memory allocation with LCOW/windows containers? (#1357) @SteveTownsendBway<https://github.com/SteveTownsendBway> We plan to use docker-compose to run our Linux containers via LCOW. Will moby/moby#37296<moby/moby#37296> fix this issue for docker-compose too? Will the docker-compose mem_reservation: flag suddenly start working again? Or will a new PR be required for docker-compose? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#1357 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AllTTPHaPUAN1z1aMJT9xNpdlZ1xPxt_ks5uxq-SgaJpZM4QqLGT>.

qbikez · 2018-11-24T21:55:46Z

Will the docker-compose mem_reservation: flag suddenly start working again? Or will a new PR be required for docker-compose?

@KwalityKoder After applying moby/moby#37296, mem_limit in docker-compose works. mem_reservation still gives an error: "invalid option: Windows does not support MemoryReservation".

- There are 2 current issues with building the Alpine based agent container in Azure: - LCOW doesn't support memory limits on containers and gcc currently gets an OOM (out of memory) error during compilation. This issue is documented in a few places, notably: docker/for-win#1357 moby/moby#37296 - Multi-stage builds aren't fully supported in LCOW, though some portions are working. Of note, the following issue is still open: microsoft/opengcs#156 And other issues have been resolved: microsoft/opengcs#168 microsoft/opengcs#169 The LCOW "epic" should eventually be updated when support is complete: moby/moby#33850 It's difficult at this time to tell if the current multi-stage support is adequate given the blocking memory problem. It may very well be enough. - For now, disable the Azure pipelines steps for building until the point at which they can be re-enabled.

docker-robott · 2019-02-22T02:00:21Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

jimpriest · 2019-02-22T14:15:30Z

/remove-lifecycle stale

sreejith-mq · 2019-03-11T17:20:07Z

As other people have mentioned, this is pretty much covered under moby/moby#37296. According to the latest comment on that thread, the feature is now part of docker nightly builds. However no ETA on when this will be part of edge/stable of CE or EE

docker-robott · 2019-06-09T01:00:23Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

dmastag · 2019-06-10T11:18:05Z

/remove-lifecycle stale

temuera · 2019-06-21T21:25:14Z

/remove-lifecycle stale

idontsov · 2019-07-10T14:32:30Z

Any update?

docker-robott · 2019-10-08T01:00:20Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

idontsov · 2019-10-08T09:48:00Z

/remove-lifecycle stale

idontsov · 2019-10-08T09:48:09Z

/lifecycle frozen

AAATechGuy · 2021-04-17T08:54:43Z

faced similar issue today, when trying to spawn up linux (sql) image on LCOW, via docker-desktop.

gauravshegokar · 2022-03-11T00:49:38Z

/remove-lifecycle frozen

gauravshegokar · 2022-03-11T00:49:53Z

/remove-lifecycle stale

docker-robott · 2022-06-09T01:00:07Z

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

docker-robott · 2022-08-08T03:00:12Z

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked

rlpowell mentioned this issue Nov 25, 2017

what is LCOW? Storage Driver: windowsfilter (windows) lcow (linux) ? Maybe It is a new thing? #1248

Closed

jasonbivins added area/hyper-v area/LCOW area/wincon area/windows 10 kind/bug kind/OS status/triage labels Dec 6, 2017

twright-msft mentioned this issue Jan 2, 2018

sqlservr: This program requires a machine with at least 3250 megabytes of memory. microsoft/mssql-docker#114

Closed

docker-robott added the lifecycle/stale label Jul 31, 2018

docker-robott removed the lifecycle/stale label Jul 31, 2018

docker-robott added the lifecycle/stale label Feb 22, 2019

docker-robott removed the lifecycle/stale label Feb 22, 2019

docker-robott added the lifecycle/stale label Jun 9, 2019

docker-robott removed the lifecycle/stale label Jun 10, 2019

crossan007 mentioned this issue Aug 22, 2019

Code-server sshcode all requests time out coder/code-server#912

Closed

docker-robott added the lifecycle/stale label Oct 8, 2019

docker-robott removed the lifecycle/stale label Oct 8, 2019

docker-robott added the lifecycle/frozen label Oct 8, 2019

docker-robott removed the lifecycle/frozen label Mar 11, 2022

docker-robott added the lifecycle/stale label Jun 9, 2022

docker-robott closed this as completed Jul 9, 2022

docker locked and limited conversation to collaborators Aug 8, 2022

docker-robott added the lifecycle/locked label Aug 8, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RAM/memory allocation with LCOW/windows containers? #1357

RAM/memory allocation with LCOW/windows containers? #1357

rlpowell commented Nov 24, 2017

rlpowell commented Nov 25, 2017

liquidboy commented Jan 30, 2018 •

edited

Loading

fizxmike commented Apr 24, 2018

docker-robott commented Jul 31, 2018

nerumo commented Jul 31, 2018

bloodybeet commented Aug 31, 2018

icenine457 commented Sep 14, 2018

greyphoenix commented Sep 30, 2018

pccruiser commented Oct 6, 2018

jimpriest commented Oct 7, 2018

nayala0791 commented Nov 20, 2018

KwalityKoder commented Nov 21, 2018

bloodybeet commented Nov 21, 2018 via email

KwalityKoder commented Nov 22, 2018

SteveTownsendBway commented Nov 22, 2018 •

edited

Loading

KwalityKoder commented Nov 22, 2018

DecisionSystems commented Nov 22, 2018

SteveTownsendBway commented Nov 22, 2018 via email

qbikez commented Nov 24, 2018 •

edited

Loading

docker-robott commented Feb 22, 2019

jimpriest commented Feb 22, 2019

sreejith-mq commented Mar 11, 2019

docker-robott commented Jun 9, 2019

dmastag commented Jun 10, 2019

temuera commented Jun 21, 2019

idontsov commented Jul 10, 2019

docker-robott commented Oct 8, 2019

idontsov commented Oct 8, 2019

idontsov commented Oct 8, 2019

AAATechGuy commented Apr 17, 2021 •

edited

Loading

gauravshegokar commented Mar 11, 2022

gauravshegokar commented Mar 11, 2022

docker-robott commented Jun 9, 2022

docker-robott commented Aug 8, 2022

RAM/memory allocation with LCOW/windows containers? #1357

RAM/memory allocation with LCOW/windows containers? #1357

Comments

rlpowell commented Nov 24, 2017

Expected behavior

Actual behavior

Information

Steps to reproduce the behavior

rlpowell commented Nov 25, 2017

liquidboy commented Jan 30, 2018 • edited Loading

fizxmike commented Apr 24, 2018

docker-robott commented Jul 31, 2018

nerumo commented Jul 31, 2018

bloodybeet commented Aug 31, 2018

icenine457 commented Sep 14, 2018

greyphoenix commented Sep 30, 2018

pccruiser commented Oct 6, 2018

jimpriest commented Oct 7, 2018

nayala0791 commented Nov 20, 2018

KwalityKoder commented Nov 21, 2018

bloodybeet commented Nov 21, 2018 via email

KwalityKoder commented Nov 22, 2018

SteveTownsendBway commented Nov 22, 2018 • edited Loading

KwalityKoder commented Nov 22, 2018

DecisionSystems commented Nov 22, 2018

SteveTownsendBway commented Nov 22, 2018 via email

qbikez commented Nov 24, 2018 • edited Loading

docker-robott commented Feb 22, 2019

jimpriest commented Feb 22, 2019

sreejith-mq commented Mar 11, 2019

docker-robott commented Jun 9, 2019

dmastag commented Jun 10, 2019

temuera commented Jun 21, 2019

idontsov commented Jul 10, 2019

docker-robott commented Oct 8, 2019

idontsov commented Oct 8, 2019

idontsov commented Oct 8, 2019

AAATechGuy commented Apr 17, 2021 • edited Loading

gauravshegokar commented Mar 11, 2022

gauravshegokar commented Mar 11, 2022

docker-robott commented Jun 9, 2022

docker-robott commented Aug 8, 2022

liquidboy commented Jan 30, 2018 •

edited

Loading

SteveTownsendBway commented Nov 22, 2018 •

edited

Loading

qbikez commented Nov 24, 2018 •

edited

Loading

AAATechGuy commented Apr 17, 2021 •

edited

Loading