First time here? Check out the FAQ!
0

Let wingide do the API scraping process/analysing in parallel mode.

Hi,

For a virtualenv which have many packages installed, the API scraping process/analysing will be time-consuming. Say the following for my case,

$ pip list | wc
    163     329    7630

In this case, the windide will take long time for analyzing the packages belong to the virtualenv, so I want to know whether can we let wingide utilize the multiple cores of the host for this job, i.e., done parallelly.

Regards

hongyi-zhao's avatar
497
hongyi-zhao
asked 2020-03-26 08:56:45 -0500
Wingware Support's avatar
4k
Wingware Support
updated 2020-03-26 21:44:06 -0500
edit flag offensive 0 remove flag close merge delete

Comments

add a comment see more comments

2 Answers

0

Source analysis is currently asynchronous within Wing's process, so just one core. We're moving towards moving that to separate processes so it can use more cores, but this isn't available yet. Part of the process, scraping extension module APIs by loading them does already happen in separate processes and should end up on multiple cores. But a lot of the work is analyzing Python code and not extension modules.

Wingware Support's avatar
4k
Wingware Support
answered 2020-03-26 19:32:00 -0500
edit flag offensive 0 remove flag delete link

Comments

Anything, if possible, should be done using more cores, IMO.

hongyi-zhao's avatar hongyi-zhao (2020-03-26 20:08:29 -0500) edit

Yes, although there will be a preference to control how many cores are used because people won't necessarily want Wing loading them all down if they are also working on other things.

Wingware Support's avatar Wingware Support (2020-03-26 21:38:41 -0500) edit

Agree with you.

hongyi-zhao's avatar hongyi-zhao (2020-03-30 23:47:47 -0500) edit
add a comment see more comments
0

Wing doesn't currently have a package manager so I'm a bit unclear on what you're asking, or perhaps this question is off-topic since it's about pip and not Wing.

In any case, I think if you run pip it just uses one core. The only things I could find about this by searching seem to indicate that is still how it works.

We do hope to add a package manger to Wing eventually, and will be sure we consider that this can be slow.

Wingware Support's avatar
4k
Wingware Support
answered 2020-03-26 09:35:37 -0500
edit flag offensive 0 remove flag delete link

Comments

Sorry for my phrasing, I've changed the title for this question. In fact, I mean the API scraping process/analyzing done by wingide when a new virtualenv added into wingide.

hongyi-zhao's avatar hongyi-zhao (2020-03-26 19:19:59 -0500) edit
add a comment see more comments

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss.

Add Answer