python - How to efficiently serialize a scikit-learn classifier -

-

what's efficient way serialize scikit-learn classifier?

i'm using python's standard pickle module serialize text classifier, results in monstrously large pickle. serialized object can 100mb or more, seems excessive , takes while generate , store. i've done similar work weka, , equivalent serialized classifier couple of mbs.

is scikit-learn possibly caching training data, or other extraneous info, in pickle? if so, how can speed , reduce size of serialized scikit-learn classifiers?

classifier = pipeline([     ('vectorizer', countvectorizer(ngram_range=(1,4))),     ('tfidf', tfidftransformer()),     ('clf', onevsrestclassifier(linearsvc())), ])

for large text datasets, use hashing trick: replace tfidfvectorizer hashingvectorizer (potentially stacked tfidftransformer in pipeline): faster pickle won't have store vocabulary dict more discussed in question:

how can reduce memory usage of scikit-learn vectorizers?


Comments

Post a Comment

Popular posts from this blog

c# - SelectList with Dictionary, add values to the Dictionary after it's assigned to SelectList -

-
i have following code - dictionary<string, string> mydict = new dictionary<string, string>(); mydict.add("keya", "valuea"); mydict.add("keyb", "valueb"); ienumerable<selectlistitem> myselectlist = new selectlist(mydict, "key", "value") further down in program, want add values mydict . possible? if yes, how? want - myselectlist.mydict.add("keyc", "valuec"); if you're wanting add items mydict, possible, , changes reflected in of myselectlist 's enumerations long changes made before enumeration (e.g. using .tolist() ) generated. as worked example: dictionary<string, string> mydict = new dictionary<string, string>(); mydict.add("keya", "valuea"); mydict.add("keyb", "valueb"); ienumerable<selectlistitem> myselectlist = new selectlist(mydict, "key", "value...
Read more

how can i manage url using .htaccess in php? -

-
i want manage url of website right showing me www.computermall.co.in/product_details.php?prdid=34 but want www.computermall.co.in/product_details/34 how can it? i have tried this options +followsymlinks -multiviews rewriteengine on rewritebase /computermall # uri-path directly the_request variable rewritecond %{the_request} ^(get|head)\s/(.*)\.php [nc] # strip extension , redirect permanently rewriterule .* /%2 [r=301,l,nc] rewritecond %{request_filename} !-f rewritecond %{request_filename} !-d rewritecond %{request_uri} !\.php [nc] # map internally original resource rewriterule ^(.*)$ $1.php [l,qsa] try this: rewriteengine on rewriterule ^product_details/prdid/([^/]*)$ /product_details?prdid=$1 [l] edit: removed leading slash
Read more

ios - I get the error Property '...' not found on object of type '...' -

-
appdelegate.h #import <uikit/uikit.h> @interface appdelegate : uiresponder <uiapplicationdelegate> @property (strong, nonatomic) uiwindow *window; @end appdelegate.m #import "appdelegate.h" #import "firstviewcontroller.h" #import "secondviewcontroller.h" @implementation appdelegate - (bool)application:(uiapplication *)application didfinishlaunchingwithoptions: goes on normal app.......... @end firstviewcontroller.h #import <uikit/uikit.h> @interface firstviewcontroller : uiviewcontroller @property (strong, nonatomic) iboutlet uiimageview *bground; - (ibaction)settingspressed:(id)sender; - (ibaction)startpressed:(id)sender; @end @class secondviewcontroller; @interface secondviewcontroller : uiviewcontroller @property(strong,nonatomic)secondviewcontroller *secondviewcontroller; @end firstviewcontroller.m #import "firstviewcontroller.h" @interface firstviewcontroller () @end @implement...
Read more