Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
What's new
10
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in / Register
Toggle navigation
Open sidebar
Peter-Bernd Otte
Workload Manager
Commits
0dbbaaf6
Commit
0dbbaaf6
authored
May 28, 2019
by
Peter-Bernd Otte
1
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Improve speed on worker, changed to subprocess.run call for ranks>0
parent
91115df0
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
10 additions
and
9 deletions
+10
-9
wkmgr.py
wkmgr.py
+10
-9
No files found.
wkmgr.py
View file @
0dbbaaf6
...
...
@@ -251,22 +251,23 @@ while True:
bashCommand
=
workstr
+
" > "
+
outputdir
+
str
(
jobid
)
+
"/std_out.txt 2> "
+
outputdir
+
str
(
jobid
)
+
"/err_out.txt"
try
:
p
=
subprocess
.
Popen
(
bashCommand
,
shell
=
True
)
logger
.
info
(
'Worker startet job.'
)
if
rank
==
0
:
p
=
subprocess
.
Popen
(
bashCommand
,
shell
=
True
)
# start job in background
logger
.
info
(
'Worker startet job.'
)
else
:
completed
=
subprocess
.
run
(
bashCommand
,
shell
=
True
)
# start job and wait for return
logger
.
info
(
'Job ended, return code: '
+
str
(
completed
.
returncode
)
)
except
subprocess
.
CalledProcessError
as
err
:
logger
.
info
(
'Worker '
+
str
(
rank
)
+
' ERROR:'
,
err
)
WorkerFSM
=
3
WorkerFSM
=
(
3
if
rank
==
0
else
0
)
if
WorkerFSM
==
3
:
#State 3: Running
if
WorkerFSM
==
3
:
#State 3: Running
, used only for rank = 0 with subprocess.Popen
if
p
.
poll
()
is
not
None
:
logger
.
info
(
"Job ended, ret code: "
+
str
(
p
.
returncode
))
logger
.
info
(
"Job ended, ret
urn
code: "
+
str
(
p
.
returncode
))
WorkerFSM
=
0
if
rank
==
0
:
time
.
sleep
(
0.01
)
# to take some load off the node
else
:
time
.
sleep
(
1
)
time
.
sleep
(
0.01
)
# to take some load off the node
logger
.
info
(
"Worker ended."
)
...
...
Peter-Bernd Otte
@pbotte
mentioned in issue
#1 (closed)
·
May 28, 2019
mentioned in issue
#1 (closed)
mentioned in issue #1
Toggle commit list
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment