Let wingide do the API scraping process/analysing in parallel mode.

virtualenv
pip
feature-request
retag add tags

Hi,

For a virtualenv which have many packages installed, the API scraping process/analysing will be time-consuming. Say the following for my case,

$ pip list | wc
    163     329    7630

In this case, the windide will take long time for analyzing the packages belong to the virtualenv, so I want to know whether can we let wingide utilize the multiple cores of the host for this job, i.e., done parallelly.

Regards

547

hongyi-zhao

asked 2020-03-26 08:56:45 -0500

4.1k

Wingware Support

updated 2020-03-26 21:44:06 -0500

edit flag offensive 0 remove flag close merge delete

Comments

add a comment see more comments

2 Answers

Sort by

oldest

newest

most voted

Wing doesn't currently have a package manager so I'm a bit unclear on what you're asking, or perhaps this question is off-topic since it's about pip and not Wing.

In any case, I think if you run pip it just uses one core. The only things I could find about this by searching seem to indicate that is still how it works.

We do hope to add a package manger to Wing eventually, and will be sure we consider that this can be slow.

4.1k

Wingware Support

answered 2020-03-26 09:35:37 -0500

edit flag offensive 0 remove flag delete link

Comments

Sorry for my phrasing, I've changed the title for this question. In fact, I mean the API scraping process/analyzing done by wingide when a new virtualenv added into wingide.

hongyi-zhao (2020-03-26 19:19:59 -0500) edit

add a comment see more comments

Source analysis is currently asynchronous within Wing's process, so just one core. We're moving towards moving that to separate processes so it can use more cores, but this isn't available yet. Part of the process, scraping extension module APIs by loading them does already happen in separate processes and should end up on multiple cores. But a lot of the work is analyzing Python code and not extension modules.

4.1k

Wingware Support

answered 2020-03-26 19:32:00 -0500

edit flag offensive 0 remove flag delete link

Comments

Anything, if possible, should be done using more cores, IMO.

hongyi-zhao (2020-03-26 20:08:29 -0500) edit

Yes, although there will be a preference to control how many cores are used because people won't necessarily want Wing loading them all down if they are also working on other things.

Wingware Support (2020-03-26 21:38:41 -0500) edit

Agree with you.

hongyi-zhao (2020-03-30 23:47:47 -0500) edit

add a comment see more comments

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss.

Add Answer

Let wingide do the API scraping process/analysing in parallel mode. edit

Comments

2 Answers

Comments

Comments