clips /pattern
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
★ 8,857⑂ 1,549👁 522⑃ 134● 210▤ 1
+0
Daily
8,857
stars
1,549
forks
522
watchers
134
prs
210
issues
1
releases
README
WARNING: This repository is no longer maintained.
This project is archived and will not receive further updates, bug fixes, or security patches. Issues and pull requests may not be reviewed.
Use this software at your own risk.
__________________________________________________
Pattern
=======
Pattern is a web mining module for Python. It has tools for:
Data Mining: web services (Google, Twitter, Wikipedia), web crawler, HTML DOM parser
Natural Language Processing: part-of-speech taggers, n-gram search, sentiment analysis, WordNet
Machine Learning: vector space model, clustering, classification (KNN, SVM, Perceptron)
Network Analysis: graph centrality and visualization.
It is well documented, thoroughly tested with 350+ unit tests and comes bundled with 50+ examples. The source code is licensed under BSD.
!Example workflow
Example
-------
This example trains a classifier on adjectives mined from Twitter using Python 3. First, tweets that contain hashtag win or fail are collected. For example: "$20 tip off a sweet little old lady today win". The word part-of-speech tags are then parsed, keeping only adjectives. Each tweet is transformed to a vector, a dictionary of adjective → count items, labeled WIN or FAIL. The classifier uses the vectors to learn which other tweets look more like WIN or more like FAIL.
python
from pattern.web import Twitter
from pattern.en import tag
from pattern.vector import KNN, count
twitter, knn = Twitter(), KNN()
for i in range(1, 3):
for tweet in twitter.search('win OR fail', start=i, count=100):
s = tweet.text.lower()
p = 'win' in s and 'WIN' or 'FAIL'
v = tag(s)
v = [word for word, pos in v if pos == 'JJ'] JJ = adjective
v = count(v) {'sweet': 1}
if v:
knn.train(v, type=p)
print(knn.classify('sweet potato burger'))
print(knn.classify('stupid autocorrect'))
Installation
------------
Pattern supports Python 2.7 and Python 3.6. To install Pattern so that it is available in all your scripts, unzip the download and from the command line do:
bash
cd pattern-3.6
python setup.py install
If you have pip, you can automatically download and install from the PyPI repository:
bash
pip install pattern
If none of the above works, you can make Python aware of the module in three ways:
- Put the pattern folder in the same folder as your script.
- Put the pattern folder in the standard location for modules so it is available to all scripts:
c:\python36\Lib\site-packages\ (Windows),
/Library/Python/3.6/site-packages/ (Mac OS X),
/usr/lib/python3.6/site-packages/ (Unix).
- Add the location of the module to sys.path in your script, before importing it:
python
MODULE = '/users/tom/desktop/pattern'
import sys; if MODULE not in sys.path: sys.path.append(MODULE)
from pattern.en import parsetree
Documentation
-------------
For documentation and examples see the user documentation.
Version
-------
3.6
License
-------
BSD, see LICENTop issues
#62Python 3 support70 comments · 2013-12-19#8how extend to other languages?19 comments · 2011-09-05#203SQL config error on Mac OSX installation11 comments · 2018-01-03#275Installing fails: mysql_config not found9 comments · 2019-04-22#243StopIteration raised every time 8 comments · 2018-08-09#279 Command "python setup.py egg_info" failed with error code 1 7 comments · 2019-06-26#209Error while installing7 comments · 2018-01-24#238ModuleNotFoundError: No module named 'pattern.en'6 comments · 2018-07-03
Latest release
3.7-beta 3.7-beta Pre-release 2022-08-30Beta release of version 3.7 including support for Python versions 3.6 ~ 3.10. Includes multiple fixes to code to reduce unit test noise and associated fixes to tests.