[gpaw-users] 回复： Assistance Required: Memory Consumption Issue in GPAW Calculations

Wed Sep 25 23:26:15 CEST 2024

Dear all

I hope this email finds you well.

First and foremost, thanks for Jens Jørgen's helpful suggestion.  After addressing the current issue of increasing thread memory consumption over time, I will definitely take your advice into account to optimize the number of cores.

Regarding the memory usage monitoring, I encountered a minor obstacle when using `grep MEM gpaw.out`, as it did not provide the memory usage for each step as expected. To work around this, I utilized `ps -aux | grep gpaw` and `free -h` to monitor the memory usage of gpaw threads and the device. I redirected the outputs of these commands to two separate files: process_mem.txt for gpaw threads and device_mem.txt for device memory. Due to the size exceeding the attachment limit, I have uploaded these files to Google Drive for your convenience. You can access the files using the following link: 

https://drive.google.com/drive/folders/1tNg3AiWAYXEIEzQKP8FZnQGLN4dLh0qJ?usp=drive_link

For your immediate reference, I have attached the extracted data files and the plotting script to this email. The attached graph, generated using the data from these files, clearly illustrates the increase in memory usage over time.

Furthermore, in my recent testing, I noticed that when the trajectory parameter was not specified during the execution of opt.run, the memory usage did not increase as it had previously. I am curious to learn more about the reason for this change.

Thank you once again for your valuable assistance.

Best regards,

Wei Cao

------------------ 原始邮件 ------------------
发件人:                                                                                                                        "Georg Kastlunger"                                                                                    <geokast at dtu.dk>;
发送时间: 2024年9月24日(星期二) 晚上11:24
收件人: "Jens Jørgen Mortensen"<jjmo at dtu.dk>;"Adil"<weicaocn at qq.com>;"gpaw-users"<gpaw-users at listserv.fysik.dtu.dk>;
抄送: "Peterson, Andrew"<Andrew_Peterson at brown.edu>;"Melander, Marko"<marko.m.melander at jyu.fi>;
主题: Re: [gpaw-users] Assistance Required: Memory Consumption Issue in GPAW Calculations

Dear Wei,

I tried to reproduce your error with your input but couldn’t. The memory usage for me using your input scripts stayed quite constant.

Can you try to produce the same plot, on your machine using the attached input files? The memory usage will be accessible via “grep MEM gpaw.out” after the calculation finished.

Best wishes,

Georg

From:Jens Jørgen Mortensen <jjmo at dtu.dk>
Date: Tuesday, 24 September 2024 at 10.41
To: Adil <weicaocn at qq.com>, gpaw-users <gpaw-users at listserv.fysik.dtu.dk>
Cc: Georg Kastlunger <geokast at dtu.dk>, Peterson, Andrew <Andrew_Peterson at brown.edu>, Melander, Marko <marko.m.melander at jyu.fi>
Subject: Re: [gpaw-users] Assistance Required: Memory Consumption Issue in GPAW Calculations

On 9/21/24 21:19, Adil via gpaw-users wrote:
> Dear gpaw developers
> 
> I hope this message finds you well. I am writing to seek your expertise 
> regarding an issue I've encountered while performing calculations with 
> GPAW.I meet a question about memory consumption recently when I try to 
> optimize the geometry of a 80-atoms slab.
> 
> The question arise because I find the memory consumption of this task is 
> extremely high. When I try to use /--dry-run/ to estimate the memory, 
> the anticipated memory consumption was around 500 MiB. However, during 
> the actual computation, each process consumed a minimum of 1 GiB of 
> memory, and it continued to escalate as the calculation progressed. 
> After ten steps of optimization, the memory usage even alarmingly 
> reached 7 GiB in some processes. By the end of the computation, it even 
> peaked at a single-threaded maximum of 27 GiB.
> 
> As for the parallel parameters, I have taken note of the documentation 
> which states that if parameters such as domain are not specified, the 
> calculator will automatically select an appropriate value. Given this, I 
> have decided against manually setting these parameters to avoid any 
> potential misconfiguration.
> 
> Additionally, I have attached my computation script and output files for 
> your comprehensive review.
> 
> Could you please provide some guidance on whether this level of memory 
> consumption is expected for calculations of this scale and complexity? 
> If it is not, I would appreciate any recommendations on how to optimize 
> the memory usage or any potential modifications to the calculation 
> parameters that might help.
> 
> I understand that memory management can be a complex issue, and any 
> insights you can offer would be invaluable. If there are any specific 
> logs or additional information you need from me to better diagnose the 
> problem, please let me know.
> 
> Thank you in advance for your time and assistance. I look forward to 
> your response.

I'm not so familiar with SJM calculations.  Georg, Andrew, Marko: do you 
have any comments?

I can see that you have 5 IBZ k-points, so you could choose a number of 
cores that is divisible by 5 - that should reduce memory use.

Jens Jørgen

> Best regards,
> Wei Cao
> Jinan University, China
> 
> _______________________________________________
> gpaw-users mailing list
> gpaw-users at listserv.fysik.dtu.dk
> https://listserv.fysik.dtu.dk/mailman/listinfo/gpaw-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.fysik.dtu.dk/pipermail/gpaw-users/attachments/20240926/b6efe625/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 8C08719B at CEFB6902.F77FF466.jpg
Type: image/jpeg
Size: 0 bytes
Desc: not available
URL: <http://listserv.fysik.dtu.dk/pipermail/gpaw-users/attachments/20240926/b6efe625/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 950A890D at 8F946C79.F77FF466.png
Type: application/octet-stream
Size: 51882 bytes
Desc: not available
URL: <http://listserv.fysik.dtu.dk/pipermail/gpaw-users/attachments/20240926/b6efe625/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Mem_list.7z
Type: application/octet-stream
Size: 22449 bytes
Desc: not available
URL: <http://listserv.fysik.dtu.dk/pipermail/gpaw-users/attachments/20240926/b6efe625/attachment-0003.obj>