Comments on: Building a Vector Space Search Engine in Python

By: JS

JS — Mon, 08 Dec 2008 20:26:19 +0000

If the query contains unseen terms in training documents, your code will break. For example, if the query was “cat shannon”, i.e. vectorSpace.search(["cat", "shannon"])), the “vector[self.vectorKeywordIndex[word]]” line of makeVector will break since “shannon” is not indexed.

By: Giuliani Vito, Ivan

Giuliani Vito, Ivan — Sat, 06 Dec 2008 15:44:22 +0000

You might be interested in django-searchable (http://code.google.com/p/django-searchable). It is a django application that uses a vector space model with tf-idf weighting. It’s experimental stuff but you can already see how tf-idf weighting works really.

By: Joseph Wilk

Joseph Wilk — Thu, 13 Nov 2008 21:25:15 +0000

Thanks for spotting the error. There was a problem with the PorterStemmer.py having been truncated. I've updated the version at Github (http://github.com/josephwilk/semanticpy/tree/master) and that works now.

By: ronin1770

ronin1770 — Thu, 13 Nov 2008 11:43:57 +0000

hi,

i am trying to run this code – however i am getting the following error

Traceback (most recent call last):
File “E:\TayaIT_Dev\vector_model\VectorSpace.py”, line 93, in
vectorSpace= VectorSpace(documents)
File “E:\TayaIT_Dev\vector_model\VectorSpace.py”, line 25, in __init__
self.build(documents)
File “E:\TayaIT_Dev\vector_model\VectorSpace.py”, line 30, in build
self.vectorKeywordIndex = self.getVectorKeywordIndex(documents)
File “E:\TayaIT_Dev\vector_model\VectorSpace.py”, line 41, in getVectorKeywordIndex
vocabularyList = self.parser.tokenise(vocabularyString)
File “E:\TayaIT_Dev\vector_model\Parser.py”, line 36, in tokenise
return [self.stemmer.stem(word,0,len(word)-1) for word in words]
AttributeError: PorterStemmer instance has no attribute ’stem’

any ideas?

thanx in advance

By: links for 2008-07-03 « Breyten’s Dev Blog

links for 2008-07-03 « Breyten’s Dev Blog — Thu, 03 Jul 2008 11:32:38 +0000

[...] Building a Vector Space Search Engine in Python (tags: code python search) [...]

By: dineshv

dineshv — Fri, 30 May 2008 16:12:02 +0000

Wrt my previous note the download zip file only included a portion of the PorterStemmer.py file. Anyway, I was just interested in a Python stemming solution which I got from http://tartarus.org/~martin/PorterStemmer/python.txt. Cheers!

Dinesh