recent issue with circle-ci

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

recent issue with circle-ci

Zouhair Mahboubi

Hi,

It looks some urls and installation instructions have changed recently? I had a gitlab-ci that started failing:

E: The repository 'http://downloads.skewed.de/apt/buster buster Release' does not have a Release file.

I tried following the new instructions to update the tests, and while apt-get is not complaining anymore and the installation appears to be successful, when the tests run it is not able to find graph_tool. Here is a MWE that’s failing: (I replaced my usual pytest with a simple attempt to import the library to illustrate)
(Note I’m using add-apt-repository. Previously I was using add-apt-repository -s but it seems like having the deb-src was breaking things)

test:
  image: python:3.6-slim
  stage: test
  before_script:
    - apt-get update
    - apt-get install -y gnupg2 software-properties-common
    - apt-key adv --keyserver keys.openpgp.org --recv-key 612DEFB798507F25
    - add-apt-repository "deb http://downloads.skewed.de/apt buster main"
    - apt-get update
    - apt-get install -y gcc python3-dev python3-pip
    - apt-get install -y python3-graph-tool
    # - pip3 install -r requirements.txt
    - pip3 freeze
    - python3 -c 'from graph_tool.all import *'

Here is a snippet from the output. I didn’t see any error during installation (tried this both on gitlab and locally with gitlab-runner)

$ apt-get install -y python3-graph-tool
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  adwaita-icon-theme at-spi2-core blt dbus-user-session
  dconf-gsettings-backend dconf-service fontconfig fontconfig-config
  ...
Need to get 115 MB of archives.
After this operation, 575 MB of additional disk space will be used.
 ...
Unpacking python3-graph-tool (2.31) ...
Setting up libxdot4 (2.40.1-6) ...
Setting up javascript-common (11) ... 
...

$ pip3 freeze
$ python3 -c 'from graph_tool.all import *'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'graph_tool'
Running after script...
ERROR: Job failed: exit code 1
FATAL: exit code 1

_______________________________________________
graph-tool mailing list
[hidden email]
https://lists.skewed.de/mailman/listinfo/graph-tool
Reply | Threaded
Open this post in threaded view
|

Re: recent issue with circle-ci

Tiago Peixoto
Administrator
Hi,

Indeed the apt repository has changed, due to a re-organization of the
package builds using CI.

However, I cannot reproduce the problem you are encountering. Are you
sure you are using the standard distribution python version?

Best,
Tiago

Am 30.03.20 um 03:37 schrieb Zouhair Mahboubi:

> Hi,
>
> It looks some urls and installation instructions have changed recently?
> I had a gitlab-ci that started failing:
>
> |E: The repository 'http://downloads.skewed.de/apt/buster buster
> Release' does not have a Release file. |
>
> I tried following the new instructions to update the tests, and while
> apt-get is not complaining anymore and the installation appears to be
> successful, when the tests run it is not able to find graph_tool. Here
> is a MWE that’s failing: (I replaced my usual pytest with a simple
> attempt to import the library to illustrate)
> (Note I’m using |add-apt-repository|. Previously I was using
> |add-apt-repository -s| but it seems like having the deb-src was
> breaking things)
>
> |test: image: python:3.6-slim stage: test before_script: - apt-get
> update - apt-get install -y gnupg2 software-properties-common - apt-key
> adv --keyserver keys.openpgp.org <http://keys.openpgp.org> --recv-key
> 612DEFB798507F25 - add-apt-repository "deb
> http://downloads.skewed.de/apt buster main" - apt-get update - apt-get
> install -y gcc python3-dev python3-pip - apt-get install -y
> python3-graph-tool # - pip3 install -r requirements.txt - pip3 freeze -
> python3 -c 'from graph_tool.all import *' |
>
> Here is a snippet from the output. I didn’t see any error during
> installation (tried this both on gitlab and locally with gitlab-runner)
>
> |$ apt-get install -y python3-graph-tool Reading package lists...
> Building dependency tree... Reading state information... The following
> additional packages will be installed: adwaita-icon-theme at-spi2-core
> blt dbus-user-session dconf-gsettings-backend dconf-service fontconfig
> fontconfig-config ... Need to get 115 MB of archives. After this
> operation, 575 MB of additional disk space will be used. ... Unpacking
> python3-graph-tool (2.31) ... Setting up libxdot4 (2.40.1-6) ... Setting
> up javascript-common (11) ... ... $ pip3 freeze $ python3 -c 'from
> graph_tool.all import *' Traceback (most recent call last): File
> "<string>", line 1, in <module> ModuleNotFoundError: No module named
> 'graph_tool' Running after script... ERROR: Job failed: exit code 1
> FATAL: exit code 1 |
>
>
> _______________________________________________
> graph-tool mailing list
> [hidden email]
> https://lists.skewed.de/mailman/listinfo/graph-tool
>

--
Tiago de Paula Peixoto <[hidden email]>


_______________________________________________
graph-tool mailing list
[hidden email]
https://lists.skewed.de/mailman/listinfo/graph-tool

signature.asc (849 bytes) Download Attachment
--
Tiago de Paula Peixoto <tiago@skewed.de>
Reply | Threaded
Open this post in threaded view
|

Re: recent issue with circle-ci

Zouhair Mahboubi
In reply to this post by Zouhair Mahboubi

Thanks Tiago,
This is not an issue I’m seeing on my local machine (I’m on OSX) since I haven’t had to reinstall graph_tool. This is only failing within Docker which is relevant since gitlab uses it for CI. Aside from the change to the apt-url, the script I included worked fine before but is now failing.

I tried with the following Dockerfile to not have to depend on gitlab and so that it’s easier to reproduce (can use play-with-docker)

FROM python:3.7-slim

RUN apt-get update &&\
    apt-get install -y gnupg2 software-properties-common &&\
    apt-key adv --keyserver keys.openpgp.org --recv-key 612DEFB798507F25 &&\
    add-apt-repository "deb http://downloads.skewed.de/apt buster main" &&\
    apt-get update &&\
    apt-get install -y python3-graph-tool &&\
    pip3 install numpy &&\
    pip3 freeze &&\
    python3 -c 'from numpy import *'  &&\
    python3 -c 'from graph_tool.all import *'

This is the output of the last three lines:

Installing collected packages: numpy
Successfully installed numpy-1.18.2
numpy==1.18.2
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'graph_tool'

you can see that pip3 freeze is only showing numpy, which when imported is fine, but importing graph_tool fails. I ran the docker image to inspect, and this is the problem I think: it looks like that doing apt-get install -y python3-graph-tool is installing it for the /usr/bin/python3.7, but the python (and pip) versions visible in the PATH to docker is in /usr/local/bin. I am not sure why this would be behaving differently now…

docker build . -t test
docker run -it --rm test /bin/bash

root@afe35d645652:/# which python3
/usr/local/bin/python3
root@afe35d645652:/# python3
Python 3.7.7 (default, Mar 11 2020, 00:35:40) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import graph_tool
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'graph_tool'
>>> 
root@afe35d645652:/# /usr/bin/python
python             python2.7          python3-config     python3.7-config   python3.7m-config  python3m-config
python2            python3            python3.7          python3.7m         python3m           
root@afe35d645652:/# /usr/bin/python
python             python2.7          python3-config     python3.7-config   python3.7m-config  python3m-config
python2            python3            python3.7          python3.7m         python3m           
root@afe35d645652:/# /usr/bin/python3.7
Python 3.7.3 (default, Dec 20 2019, 18:57:59) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import graph_tool
>>>



_______________________________________________
graph-tool mailing list
[hidden email]
https://lists.skewed.de/mailman/listinfo/graph-tool
Reply | Threaded
Open this post in threaded view
|

Re: recent issue with circle-ci

Tiago Peixoto
Administrator
Am 31.03.20 um 18:01 schrieb Zouhair Mahboubi:
> you can see that pip3 freeze is only showing numpy, which when imported
> is fine, but importing graph_tool fails. I ran the docker image to
> inspect, and this is the problem I think: it looks like that doing
> |apt-get install -y python3-graph-tool| is installing it for the
> /usr/bin/python3.7, but the python (and pip) versions visible in the
> PATH to docker is in /usr/local/bin. I am not sure why this would be
> behaving differently now…

You already identified the problem. The Debian package for graph-tool
installs it in the system version of python (in /usr), as it should, but
you are using a different Python installation (in /usr/local). This is
not a bug. If you want to use parallel python installation, you need to
install packages for that, like you did using pip for numpy.

I suggest you use the debian:buster docker image, not python:3.7-slim,
since they have the same underlying Python version and things will
behave more as you are expecting.

Best,
Tiago

--
Tiago de Paula Peixoto <[hidden email]>


_______________________________________________
graph-tool mailing list
[hidden email]
https://lists.skewed.de/mailman/listinfo/graph-tool

signature.asc (849 bytes) Download Attachment
--
Tiago de Paula Peixoto <tiago@skewed.de>