Unable to build examples (Python 3.8) #1238

Cynical-Optimist · 2020-10-14T03:18:21Z

See original issue on GitLab
In GitLab by [Gitlab user @albfan] on Dec 11, 2019, 19:33

Summary

Cannot build examples

Steps to reproduce

Using pip stable release:

$ bst --version
1.4.1
$ cd doc/examples/flatpak-autotools
$ bst build hello.bst
[--:--:--][][] STATUS  Cache usage recomputed: 125.2M / infinity (0%)
[--:--:--][][] START   Build
[--:--:--][][] START   Loading elements
[00:00:00][][] SUCCESS Loading elements
[--:--:--][][] START   Resolving elements
[00:00:00][][] SUCCESS Resolving elements
[--:--:--][][] START   Resolving cached state
[00:00:00][][] SUCCESS Resolving cached state
[--:--:--][][] START   Checking sources
[00:00:00][][] SUCCESS Checking sources
 
BuildStream Version 1.4.1
  Session Start: Wednesday, 11-12-2019 at 19:15:17
  Project:       flatpak-autotools (/home/alberto/projects/python/buildstream/doc/examples/flatpak-autotools)
  Targets:       hello.bst
  Cache Usage:   125.2M / infinity (0%)
 
User Configuration
  Configuration File:      /home/alberto/.config/buildstream.conf
  Log Files:               /home/alberto/.cache/buildstream/logs
  Source Mirrors:          /home/alberto/.cache/buildstream/sources
  Build Area:              /home/alberto/.cache/buildstream/build
  Artifact Cache:          /home/alberto/.cache/buildstream/artifacts
  Strict Build Plan:       Yes
  Maximum Fetch Tasks:     10
  Maximum Build Tasks:     2
  Maximum Push Tasks:      4
  Maximum Network Retries: 2
 
Project Options
  arch: x86_64
 
Pipeline
fetch needed 1fb677ea63be533ff4391db8c764030a80500c8f1feba0959eb747f720fd3748 base/sdk.bst
     waiting 6fbcd70444b9b3bb74e48bf930fd93a0715e854bfe370c4a43b7e9d2d81caee7 base/usrmerge.bst
     waiting 60ef22ffacf88b1fb99d3bf72752456b55c83ec86672310bcf7917f55ed7d485 base.bst
     waiting 8c609e7d39e81c68c2f8162a19440d8a947f1b95535dfb18d43b7fbef611866f hello.bst
===============================================================================
[--:--:--][][] BUG     no running event loop
 
    Traceback (most recent call last):
      File "/usr/bin/bst", line 8, in <module>
        sys.exit(cli())
      File "/usr/lib/python3.8/site-packages/click/core.py", line 764, in __call__
        return self.main(*args, **kwargs)
      File "/usr/lib/python3.8/site-packages/buildstream/_frontend/cli.py", line 173, in override_main
        original_main(self, args=args, prog_name=prog_name, complete_var=None,
      File "/usr/lib/python3.8/site-packages/click/core.py", line 717, in main
        rv = self.invoke(ctx)
      File "/usr/lib/python3.8/site-packages/click/core.py", line 1137, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/usr/lib/python3.8/site-packages/click/core.py", line 956, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/usr/lib/python3.8/site-packages/click/core.py", line 555, in invoke
        return callback(*args, **kwargs)
      File "/usr/lib/python3.8/site-packages/click/decorators.py", line 27, in new_func
        return f(get_current_context().obj, *args, **kwargs)
      File "/usr/lib/python3.8/site-packages/buildstream/_frontend/cli.py", line 340, in build
        app.stream.build(elements,
      File "/usr/lib/python3.8/site-packages/buildstream/_stream.py", line 211, in build
        self._run()
      File "/usr/lib/python3.8/site-packages/buildstream/_stream.py", line 1022, in _run
        _, status = self._scheduler.run(self.queues)
      File "/usr/lib/python3.8/site-packages/buildstream/_scheduler/scheduler.py", line 153, in run
        self._sched()
      File "/usr/lib/python3.8/site-packages/buildstream/_scheduler/scheduler.py", line 427, in _sched
        self._sched_queue_jobs()
      File "/usr/lib/python3.8/site-packages/buildstream/_scheduler/scheduler.py", line 401, in _sched_queue_jobs
        self._spawn_job(job)
      File "/usr/lib/python3.8/site-packages/buildstream/_scheduler/scheduler.py", line 281, in _spawn_job
        job.spawn()
      File "/usr/lib/python3.8/site-packages/buildstream/_scheduler/jobs/job.py", line 162, in spawn
        self._watcher.add_child_handler(self._process.pid, self._parent_child_completed)
      File "/usr/lib/python3.8/asyncio/unix_events.py", line 1286, in add_child_handler
        loop = events.get_running_loop()
    RuntimeError: no running event loop

Trying with master:

$ cd buildstream
$ sudo pip3 install -e .
$ bst --version
1.91.2+286.ga8fb5cd7
 
cd doc/examples/autotools
$ bst build hello.bst
 
Error loading user configuration: /home/alberto/projects/python/buildstream/src/buildstream/data/userconfig.yaml: Severely malformed YAML:
 
'NoneType' object is not callable

What is the current bug behavior?

build fails on 1.4.1 and master

What is the expected correct behavior?

be able to build examples

Relevant logs and/or screenshots

Anything else is needed?

Possible fixes

Other relevant information

BuildStream version affected: /milestone %BuildStream_v1.4

The text was updated successfully, but these errors were encountered:

Cynical-Optimist · 2020-10-14T03:18:24Z

In GitLab by [Gitlab user @jjardon] on Dec 12, 2019, 02:30

[Gitlab user @albfan] are the examples working with bst-1.4 when using the bst-1 branch?

Cynical-Optimist · 2020-10-14T03:18:26Z

In GitLab by [Gitlab user @juergbi] on Dec 12, 2019, 05:24

There is an incompatibility with Python 3.8 that we're looking into. Related issue: #1173

Cynical-Optimist · 2020-10-14T03:18:29Z

In GitLab by [Gitlab user @juergbi] on Dec 12, 2019, 05:24

changed title from Unable to build examples to Unable to build examples{+ (Python 3.8)+}

Cynical-Optimist · 2020-10-14T03:18:32Z

In GitLab by [Gitlab user @albfan] on Dec 12, 2019, 07:29

[Gitlab user @jjardon] I checkout origin/bst-1 and results are the same as for tag 1.4.1

[Gitlab user @juergbi] I forget to add that info:

$ python --version
Python 3.8.0

So yes, maybe this is related with python 3.8. I went through #1173, related issues and MR but didn't find what I can cherry-pick to make bst to work with python 3.8 (building from source), any tips?

I tried with a virtualenv using a local python 3.7.5 and results for bst build hello.bst are the same. Not sure if there's anything else needed in Buildstream to work from virtualenv.

If there's anything I can provide feedback, just let me know.

I'm just revisiting my main use case for buildstream: Trying to build a vala project based on flatpak Elementary SDK, and a C project dependency, to work at same time in vala project and C project.

Cynical-Optimist · 2020-10-14T03:18:35Z

In GitLab by [Gitlab user @BenjaminSchubert] on Dec 12, 2019, 16:35

[Gitlab user @albfan]

For python3.8: https://gitlab.com/BuildStream/buildstream/commit/e188b16d7979f75823200c5104b3a5fa952fc50c#65c9089123965199008f5d09bc3e5d13b56777dd_174_174 the changes in this file should be able to fix the immediate problem.

Please note that this still has some problems, but they seem to be related to how we run tests with coverage.

I tried with a virtualenv using a local python 3.7.5 and results for bst build hello.bst are the same. Not sure if there's anything else needed in Buildstream to work from virtualenv.

There is nothing needed to work in virtualenvs for BuildStream, we've been using it extensively.
Could you post the output of which bst and echo $VIRTUAL_ENV ?

Cynical-Optimist · 2020-10-14T03:18:38Z

In GitLab by [Gitlab user @albfan] on Dec 12, 2019, 17:15

[Gitlab user @BenjaminSchubert] About virtualenv: I tried again and now it's working for autotools example.

For flatpak-autotools:

$ bst build hello.bst
[--:--:--][][] START   Build
[--:--:--][][] START   Loading elements
[00:00:00][][] SUCCESS Loading elements
[--:--:--][][] START   Resolving elements
[00:00:00][][] SUCCESS Resolving elements
[--:--:--][][] START   Resolving cached state
[00:00:00][][] SUCCESS Resolving cached state
[--:--:--][][] START   Checking sources
[00:00:00][][] SUCCESS Checking sources

BuildStream Version 1.4.1
  Session Start: Thursday, 12-12-2019 at 18:14:10
  Project:       flatpak-autotools (/home/alberto/projects/python/buildstream/doc/examples/flatpak-autotools)
  Targets:       hello.bst
  Cache Usage:   125.3M / infinity (0%)

User Configuration
  Configuration File:      /home/alberto/.config/buildstream.conf
  Log Files:               /home/alberto/.cache/buildstream/logs
  Source Mirrors:          /home/alberto/.cache/buildstream/sources
  Build Area:              /home/alberto/.cache/buildstream/build
  Artifact Cache:          /home/alberto/.cache/buildstream/artifacts
  Strict Build Plan:       Yes
  Maximum Fetch Tasks:     10
  Maximum Build Tasks:     2
  Maximum Push Tasks:      4
  Maximum Network Retries: 2

Project Options
  arch: x86_64

Pipeline
fetch needed 6afb679221210681d694cb1e2a160460e4e739df836f5648216329b539909115 base/sdk.bst 
     waiting d07b99cd5e1dec6b89ac0218d1cec80a7cdfae507caf0c55194143a39fda7814 base/usrmerge.bst 
     waiting 339e2946759cc32b3545273ab9e62d810fe89a48a825247c0c141a7d3d63bc20 base.bst 
     waiting 6284d7c804cd409ee9596bf568b4b39dd263681b96c6e4e9a44fd35b3174c1ea hello.bst 
===============================================================================
[--:--:--][6afb6792][fetch:base/sdk.bst                  ] START   flatpak-autotools/base-sdk/6afb6792-fetch.197125.log
[00:00:00][6afb6792][fetch:base/sdk.bst                  ] BUG     Fetch

    An unhandled exception occured:
    
    Traceback (most recent call last):
      File "/home/alberto/projects/python/buildstream/virtualenv/buildstream-ve/lib/python3.7/site-packages/buildstream/_scheduler/jobs/job.py", line 413, in _child_action
        result = self.child_process()
      File "/home/alberto/projects/python/buildstream/virtualenv/buildstream-ve/lib/python3.7/site-packages/buildstream/_scheduler/jobs/elementjob.py", line 94, in child_process
        return self._action_cb(self._element)
      File "/home/alberto/projects/python/buildstream/virtualenv/buildstream-ve/lib/python3.7/site-packages/buildstream/_scheduler/queues/fetchqueue.py", line 46, in process
        source._fetch(previous_sources)
      File "/home/alberto/projects/python/buildstream/virtualenv/buildstream-ve/lib/python3.7/site-packages/buildstream/source.py", line 695, in _fetch
        self.__do_fetch()
      File "/home/alberto/projects/python/buildstream/virtualenv/buildstream-ve/lib/python3.7/site-packages/buildstream/source.py", line 1014, in __do_fetch
        self.fetch(**kwargs)
      File "/home/alberto/projects/python/buildstream/virtualenv/buildstream-ve/lib/python3.7/site-packages/buildstream/plugins/sources/ostree.py", line 126, in fetch
        remote_name = self.ensure_remote(self.url)
      File "/home/alberto/projects/python/buildstream/virtualenv/buildstream-ve/lib/python3.7/site-packages/buildstream/plugins/sources/ostree.py", line 194, in ensure_remote
        _ostree.configure_remote(self.repo, remote_name, url, key_url=gpg_key)
      File "/home/alberto/projects/python/buildstream/virtualenv/buildstream-ve/lib/python3.7/site-packages/buildstream/_ostree.py", line 274, in configure_remote
        repo.remote_gpg_import(remote, stream, None, 0, None)
    TypeError: OSTree.Repo.remote_gpg_import() takes exactly 5 arguments (6 given)

Not sure what should be the correct ostree version here.

I will try with patch too to see if I can send feedback.

Cynical-Optimist · 2020-10-14T03:18:41Z

In GitLab by [Gitlab user @BenjaminSchubert] on Dec 12, 2019, 17:35

That problem's https://gitlab.com/BuildStream/buildstream/issues/1164. Lets move this part of the discussion there if needed :)

Cynical-Optimist · 2020-10-14T03:18:43Z

In GitLab by [Gitlab user @albfan] on Dec 12, 2019, 22:04

Thanks for guiding. I can cherry-pick here and there.

Let's close this and contribute where is needed.

Thanks again.

Cynical-Optimist · 2020-10-14T03:18:46Z

In GitLab by [Gitlab user @albfan] on Dec 12, 2019, 22:04

closed

Cynical-Optimist closed this as completed Oct 14, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to build examples (Python 3.8) #1238

Unable to build examples (Python 3.8) #1238

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Unable to build examples (Python 3.8) #1238

Unable to build examples (Python 3.8) #1238

Comments

Cynical-Optimist commented Oct 14, 2020

Summary

Steps to reproduce

What is the current bug behavior?

What is the expected correct behavior?

Relevant logs and/or screenshots

Possible fixes

Other relevant information

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020

Cynical-Optimist commented Oct 14, 2020