Ask Your Question

Let wingide do the API scraping process/analysing in parallel mode.

asked 2020-03-26 08:56:45 -0600

hongyi-zhao's avatar

updated 2020-03-26 21:44:06 -0600


For a virtualenv which have many packages installed, the API scraping process/analysing will be time-consuming. Say the following for my case,

$ pip list | wc
    163     329    7630

In this case, the windide will take long time for analyzing the packages belong to the virtualenv, so I want to know whether can we let wingide utilize the multiple cores of the host for this job, i.e., done parallelly.


edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted

answered 2020-03-26 19:32:00 -0600

Source analysis is currently asynchronous within Wing's process, so just one core. We're moving towards moving that to separate processes so it can use more cores, but this isn't available yet. Part of the process, scraping extension module APIs by loading them does already happen in separate processes and should end up on multiple cores. But a lot of the work is analyzing Python code and not extension modules.

edit flag offensive delete link more


Anything, if possible, should be done using more cores, IMO.

hongyi-zhao's avatar hongyi-zhao  ( 2020-03-26 20:08:29 -0600 )edit

Yes, although there will be a preference to control how many cores are used because people won't necessarily want Wing loading them all down if they are also working on other things.

Wingware Support's avatar Wingware Support  ( 2020-03-26 21:38:41 -0600 )edit

Agree with you.

hongyi-zhao's avatar hongyi-zhao  ( 2020-03-30 23:47:47 -0600 )edit

answered 2020-03-26 09:35:37 -0600

Wing doesn't currently have a package manager so I'm a bit unclear on what you're asking, or perhaps this question is off-topic since it's about pip and not Wing.

In any case, I think if you run pip it just uses one core. The only things I could find about this by searching seem to indicate that is still how it works.

We do hope to add a package manger to Wing eventually, and will be sure we consider that this can be slow.

edit flag offensive delete link more


Sorry for my phrasing, I've changed the title for this question. In fact, I mean the API scraping process/analyzing done by wingide when a new virtualenv added into wingide.

hongyi-zhao's avatar hongyi-zhao  ( 2020-03-26 19:19:59 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools



Asked: 2020-03-26 08:56:45 -0600

Seen: 392 times

Last updated: Mar 26 '20