Freeze using SIRState

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Freeze using SIRState

Ronaldo Alves
Hi everyone,

I am conducting tests where I perform several parallel tests of the
SIR model (they are not using n iterations because I need to find out
when the model stabilizes, where si (t) = 0).
Parallel execution has been carried out with the multiprocessing
library. In larger networks (2000 nodes), when performing the
procedure of removing parallel edges (the graphs are converted to
undirected), the execution freeze. Can you imagine the reason for this
behavior? If I don't execute the remove_parallel_edges function,
everything happens normally ...
Execution hangs when iterate_sync will be performed. A serial run
works normally (without multiprocessing).

Would anyone have a tip?

Best regards,

Ronaldo Alves
_______________________________________________
graph-tool mailing list
[hidden email]
https://lists.skewed.de/mailman/listinfo/graph-tool
Reply | Threaded
Open this post in threaded view
|

Re: Freeze using SIRState

Tiago Peixoto
Administrator
Dear Ronaldo,

When asking questions like this, it is imperative to include a minimal
working example that reproduces the problem. Otherwise there is not much
we can do to help.

Best,
Tiago

Am 30.06.20 um 16:07 schrieb Ronaldo Alves:

> Hi everyone,
>
> I am conducting tests where I perform several parallel tests of the
> SIR model (they are not using n iterations because I need to find out
> when the model stabilizes, where si (t) = 0).
> Parallel execution has been carried out with the multiprocessing
> library. In larger networks (2000 nodes), when performing the
> procedure of removing parallel edges (the graphs are converted to
> undirected), the execution freeze. Can you imagine the reason for this
> behavior? If I don't execute the remove_parallel_edges function,
> everything happens normally ...
> Execution hangs when iterate_sync will be performed. A serial run
> works normally (without multiprocessing).
>
> Would anyone have a tip?
>
> Best regards,
>
> Ronaldo Alves
> _______________________________________________
> graph-tool mailing list
> [hidden email]
> https://lists.skewed.de/mailman/listinfo/graph-tool
>

--
Tiago de Paula Peixoto <[hidden email]>


_______________________________________________
graph-tool mailing list
[hidden email]
https://lists.skewed.de/mailman/listinfo/graph-tool

signature.asc (849 bytes) Download Attachment
--
Tiago de Paula Peixoto <tiago@skewed.de>
Reply | Threaded
Open this post in threaded view
|

Re: Freeze using SIRState

Ronaldo Alves
Sorry about that. I tried to reduce it to the maximum so that it could
be possible to simulate this behavior.
https://pastebin.com/cyCwAX0z

If I remove line 18, everything goes normally. This behavior does not
occur on smaller networks (if reduce a number of nodes on graph,
everything is ok).

Note that cpu is not used at all.

Regards,

Ronaldo Alves

Em ter., 30 de jun. de 2020 às 11:20, Tiago de Paula Peixoto
<[hidden email]> escreveu:

>
> Dear Ronaldo,
>
> When asking questions like this, it is imperative to include a minimal
> working example that reproduces the problem. Otherwise there is not much
> we can do to help.
>
> Best,
> Tiago
>
> Am 30.06.20 um 16:07 schrieb Ronaldo Alves:
> > Hi everyone,
> >
> > I am conducting tests where I perform several parallel tests of the
> > SIR model (they are not using n iterations because I need to find out
> > when the model stabilizes, where si (t) = 0).
> > Parallel execution has been carried out with the multiprocessing
> > library. In larger networks (2000 nodes), when performing the
> > procedure of removing parallel edges (the graphs are converted to
> > undirected), the execution freeze. Can you imagine the reason for this
> > behavior? If I don't execute the remove_parallel_edges function,
> > everything happens normally ...
> > Execution hangs when iterate_sync will be performed. A serial run
> > works normally (without multiprocessing).
> >
> > Would anyone have a tip?
> >
> > Best regards,
> >
> > Ronaldo Alves
> > _______________________________________________
> > graph-tool mailing list
> > [hidden email]
> > https://lists.skewed.de/mailman/listinfo/graph-tool
> >
>
>
> --
> Tiago de Paula Peixoto <[hidden email]>
>
> _______________________________________________
> graph-tool mailing list
> [hidden email]
> https://lists.skewed.de/mailman/listinfo/graph-tool
_______________________________________________
graph-tool mailing list
[hidden email]
https://lists.skewed.de/mailman/listinfo/graph-tool
Reply | Threaded
Open this post in threaded view
|

Re: Freeze using SIRState

Tiago Peixoto
Administrator
Am 30.06.20 um 17:13 schrieb Ronaldo Alves:
> Sorry about that. I tried to reduce it to the maximum so that it could
> be possible to simulate this behavior.
> https://pastebin.com/cyCwAX0z
>
> If I remove line 18, everything goes normally. This behavior does not
> occur on smaller networks (if reduce a number of nodes on graph,
> everything is ok).
>
> Note that cpu is not used at all.

Thanks for the working example.

graph-tool uses OpenMP internally to perform computations in parallel,
and this does not mix well with the processing library, at least not if
it's using fork().

You should disable OpenMP in the beginning of your code with:

    import graph_tool as gt
    gt.openmp_set_num_threads(1)

Best,
Tiago

--
Tiago de Paula Peixoto <[hidden email]>


_______________________________________________
graph-tool mailing list
[hidden email]
https://lists.skewed.de/mailman/listinfo/graph-tool

signature.asc (849 bytes) Download Attachment
--
Tiago de Paula Peixoto <tiago@skewed.de>
Reply | Threaded
Open this post in threaded view
|

Re: Freeze using SIRState

Ronaldo Alves
Hi Thiago,

I suspected there was something between Openmp and multiprocessing,
but I didn't know how to disable it. Thank you for this trick. It
really worked.

I was very curious about whether or not removing the parallel edges
directly impact the execution.

Thank you.

Em ter., 30 de jun. de 2020 às 12:36, Tiago de Paula Peixoto
<[hidden email]> escreveu:

>
> Am 30.06.20 um 17:13 schrieb Ronaldo Alves:
> > Sorry about that. I tried to reduce it to the maximum so that it could
> > be possible to simulate this behavior.
> > https://pastebin.com/cyCwAX0z
> >
> > If I remove line 18, everything goes normally. This behavior does not
> > occur on smaller networks (if reduce a number of nodes on graph,
> > everything is ok).
> >
> > Note that cpu is not used at all.
>
> Thanks for the working example.
>
> graph-tool uses OpenMP internally to perform computations in parallel,
> and this does not mix well with the processing library, at least not if
> it's using fork().
>
> You should disable OpenMP in the beginning of your code with:
>
>     import graph_tool as gt
>     gt.openmp_set_num_threads(1)
>
> Best,
> Tiago
>
> --
> Tiago de Paula Peixoto <[hidden email]>
>
> _______________________________________________
> graph-tool mailing list
> [hidden email]
> https://lists.skewed.de/mailman/listinfo/graph-tool
_______________________________________________
graph-tool mailing list
[hidden email]
https://lists.skewed.de/mailman/listinfo/graph-tool