Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change and remove FunctionsEngine functions #129

Merged
merged 27 commits into from May 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
cf1f3da
Update delete_statement
LeMyst Apr 8, 2021
0b5115e
Update delete_item
LeMyst Apr 8, 2021
ea3d9dc
Update wbi_core.py
LeMyst Apr 8, 2021
2269640
Update wbi_core.py
LeMyst Apr 8, 2021
ada0ce7
Update wbi_core.py
LeMyst Apr 8, 2021
5a0af58
Merge branch 'master' into fix-functionsengine-functions
LeMyst Apr 10, 2021
b62417e
Merge branch 'master' into fix-functionsengine-functions
LeMyst Apr 17, 2021
5d25309
Rename functions
LeMyst Apr 17, 2021
089cd26
Update wbi_core.py
LeMyst Apr 17, 2021
5d6e29f
Remove dataframe support
LeMyst Apr 17, 2021
4a64d70
Corretly remove dataframe support
LeMyst Apr 17, 2021
a987611
Bump to v0.11.0.dev0
LeMyst Apr 17, 2021
fb6c230
Split wbi_core into exceptions and functions
LeMyst Apr 19, 2021
d59112f
Merge branch 'master' into fix-functionsengine-functions
LeMyst Apr 21, 2021
170ed75
Update wbi_functions.py
LeMyst Apr 21, 2021
bd08bca
Add 3.10 to Code Compatibility Inspection
LeMyst Apr 21, 2021
bc1fb70
Merge branch 'master' into fix-functionsengine-functions
LeMyst Apr 22, 2021
dece30b
For to Python 3.10.0-alpha.7
LeMyst Apr 29, 2021
c6a3941
Merge branch 'master' into fix-functionsengine-functions
LeMyst Apr 29, 2021
613c345
Add MediaWiki API Call example in README.md
LeMyst May 1, 2021
7486db4
Merge branch 'master' into fix-functionsengine-functions
LeMyst May 1, 2021
b3b113d
Bogus version has been removed
LeMyst May 3, 2021
2f7ca4f
I tried
LeMyst May 3, 2021
928e0c6
Merge branch 'master' into fix-functionsengine-functions
LeMyst May 4, 2021
4e8a352
Merge branch 'master' into fix-functionsengine-functions
LeMyst May 7, 2021
3325d1a
Typo in comments
LeMyst May 8, 2021
835be45
Update README.md
LeMyst May 9, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 2 additions & 4 deletions .github/workflows/python-package.yml
Expand Up @@ -4,16 +4,14 @@ on:
push:
branches: [ master ]
pull_request:
branches: '**'
branches: [ '**' ]

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [ '3.7', '3.8', '3.9' ]
# Add this when Numpy add support to Python 3.10
# python-version: [ '3.7', '3.8', '3.9', '3.10-dev' ]
python-version: [ '3.7', '3.8', '3.9', '3.10-dev' ]

steps:
- uses: actions/checkout@v2
Expand Down
3 changes: 2 additions & 1 deletion .idea/inspectionProfiles/WikibaseIntegrator.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

44 changes: 32 additions & 12 deletions README.md
Expand Up @@ -12,7 +12,8 @@
- [Using a Wikibase instance](#using-a-wikibase-instance)
- [The Core Parts](#the-core-parts)
- [wbi_core.ItemEngine](#wbi_coreitemengine)
- [wbi_core.FunctionsEngine](#wbi_corefunctionsengine)
- [wbi_functions](#wbi_functions)
- [Use MediaWiki API](#use-mediawiki-api)
- [wbi_login.Login](#wbi_loginlogin)
- [Login using OAuth1 or OAuth2](#login-using-oauth1-or-oauth2)
- [Login with a username and a password](#login-with-a-username-and-a-password)
Expand Down Expand Up @@ -119,18 +120,37 @@ There are two ways of working with Wikibase items:
* A user can work with a selected QID to specifically modify the data on the item. This requires that the user knows
what he/she is doing and should only be used with great care, as this does not perform consistency checks.

Examples below illustrate the usage of ItemEngine.
## wbi_functions ##

## wbi_core.FunctionsEngine ##

wbi_core.FunctionsEngine provides a set of static functions to request or manipulate data from MediaWiki API or SPARQL
Service.
wbi_functions provides a set of static functions to request or manipulate data from MediaWiki API or SPARQL Service.

Features:

* Minimize the number of HTTP requests for reads and writes to improve performance
* Method to easily execute [SPARQL](https://query.wikidata.org) queries on the Wikibase SPARQL endpoint.

### Use MediaWiki API ###

WikibaseIntegrator don't have functions to make API call to non-wikibase actions. You can
use `wbi_functions.mediawiki_api_call_helper()` to make a custom call.

Example to get the last two revisions of entity Q42 :

```python
from wikibaseintegrator import wbi_functions

data = {
'action': 'query',
'prop': 'revisions',
'titles': 'Q42',
'rvlimit': 2,
'rvprop': 'ids|timestamp|comment|user',
'rvslots': 'main'
}

print(wbi_functions.mediawiki_api_call_helper(data, allow_anonymous=True))
```

## wbi_login.Login ##

### Login using OAuth1 or OAuth2 ###
Expand Down Expand Up @@ -242,8 +262,8 @@ address, or the URL to your bot code repository.)

## Use Mediawiki API ##

The method `wbi_core.FunctionsEngine.mediawiki_api_call_helper()` allows you to execute MediaWiki API POST call. It
takes a mandatory data array (data) and multiple optionals parameters like a login object of type wbi_login.Login, a
The method `wbi_functions.mediawiki_api_call_helper()` allows you to execute MediaWiki API POST call. It takes a
mandatory data array (data) and multiple optionals parameters like a login object of type wbi_login.Login, a
mediawiki_api_url string if the Mediawiki is not Wikidata, a user_agent string to set a custom HTTP User Agent header,
and an allow_anonymous boolean to force authentication.

Expand All @@ -252,7 +272,7 @@ Example:
Retrieve last 10 revisions from Wikidata element Q2 (Earth):

```python
from wikibaseintegrator import wbi_core
from wikibaseintegrator import wbi_functions

query = {
'action': 'query',
Expand All @@ -261,12 +281,12 @@ query = {
'rvlimit': 10
}

print(wbi_core.FunctionsEngine.mediawiki_api_call_helper(query, allow_anonymous=True))
print(wbi_functions.mediawiki_api_call_helper(query, allow_anonymous=True))
```

## Wikibase search entities ##

The method `wbi_core.ItemEngine.get_search_results()` allows for string search in a Wikibase instance. This means that
The method `wbi_core.ItemEngine.search_entities()` allows for string search in a Wikibase instance. This means that
labels, descriptions and aliases can be searched for a string of interest. The method takes five arguments: The actual
search string (search_string), an optional server (mediawiki_api_url, in case the Wikibase instance used is not
Wikidata), an optional user_agent, an optional max_results (default 500), an optional language (default 'en'), and an
Expand All @@ -275,7 +295,7 @@ option dict_id_label to return a dict of item id and label as a result.
## Merge Wikibase items ##

Sometimes, Wikibase items need to be merged. An API call exists for that, and wbi_core implements a method accordingly.
`wbi_core.FunctionsEngine.merge_items()` takes five arguments:
`wbi_functions.merge_items()` takes five arguments:
the QID of the item which should be merged into another item (from_id), the QID of the item the first item should be
merged into (to_id), a login object of type wbi_login.Login to provide the API call with the required authentication
information, a server (mediawiki_api_url) if the Wikibase instance is not Wikidata and a flag for ignoring merge
Expand Down
1 change: 0 additions & 1 deletion requirements.txt
@@ -1,6 +1,5 @@
simplejson~=3.17.2
requests~=2.25.1
pandas~=1.2.4
mwoauth~=0.3.7
backoff~=1.10.0
pytest~=6.2.4
Expand Down
4 changes: 2 additions & 2 deletions setup.cfg
@@ -1,6 +1,6 @@
[metadata]
name = wikibaseintegrator
version = 0.10.2.dev0
version = 0.11.0.dev0
author = Myst and WikidataIntegrator authors
license = MIT
license_files = LICENSE.txt
Expand All @@ -17,6 +17,7 @@ classifiers =
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
Programming Language :: Python :: 3.10
Development Status :: 4 - Beta
Operating System :: POSIX
Operating System :: MacOS :: MacOS X
Expand All @@ -32,7 +33,6 @@ packages = wikibaseintegrator
install_requires =
simplejson
requests
pandas
mwoauth
backoff
oauthlib
Expand Down
12 changes: 6 additions & 6 deletions test/test_all.py
Expand Up @@ -4,7 +4,7 @@

import requests

from wikibaseintegrator import wbi_core, wbi_fastrun
from wikibaseintegrator import wbi_core, wbi_fastrun, wbi_functions
from wikibaseintegrator.wbi_core import MWApiError

__author__ = 'Sebastian Burgstaller-Muehlbacher'
Expand All @@ -14,13 +14,13 @@
class TestMediawikiApiCall(unittest.TestCase):
def test_all(self):
with self.assertRaises(MWApiError):
wbi_core.FunctionsEngine.mediawiki_api_call_helper(data={'format': 'json', 'action': 'wbgetentities', 'ids': 'Q42'}, mediawiki_api_url="https://www.wikidataaaaaaa.org",
max_retries=3, retry_after=1, allow_anonymous=True)
wbi_functions.mediawiki_api_call_helper(data={'format': 'json', 'action': 'wbgetentities', 'ids': 'Q42'}, mediawiki_api_url="https://www.wikidataaaaaaa.org",
max_retries=3, retry_after=1, allow_anonymous=True)
with self.assertRaises(requests.HTTPError):
wbi_core.FunctionsEngine.mediawiki_api_call_helper(data=None, mediawiki_api_url="https://httpbin.org/status/400", max_retries=3, retry_after=1, allow_anonymous=True)
wbi_functions.mediawiki_api_call_helper(data=None, mediawiki_api_url="https://httpbin.org/status/400", max_retries=3, retry_after=1, allow_anonymous=True)

test = wbi_core.FunctionsEngine.mediawiki_api_call_helper(data={'format': 'json', 'action': 'wbgetentities', 'ids': 'Q42'}, max_retries=3, retry_after=1,
allow_anonymous=True)
test = wbi_functions.mediawiki_api_call_helper(data={'format': 'json', 'action': 'wbgetentities', 'ids': 'Q42'}, max_retries=3, retry_after=1,
allow_anonymous=True)
print(test)


Expand Down
9 changes: 3 additions & 6 deletions test/test_wbi_core.py
@@ -1,6 +1,6 @@
import unittest

from wikibaseintegrator import wbi_core
from wikibaseintegrator import wbi_core, wbi_functions


class TestWbiCore(unittest.TestCase):
Expand Down Expand Up @@ -105,14 +105,14 @@ def test_label(self):
assert item.get_aliases('ak') == ['c']

def test_wd_search(self):
t = wbi_core.FunctionsEngine.get_search_results('rivaroxaban')
t = wbi_functions.search_entities('rivaroxaban')
print('Number of results: ', len(t))
self.assertIsNot(len(t), 0)

def test_item_generator(self):
items = ['Q408883', 'P715', 'Q18046452']

item_instances = wbi_core.FunctionsEngine.generate_item_instances(items=items)
item_instances = wbi_functions.generate_item_instances(items=items)

for qid, item in item_instances:
self.assertIn(qid, items)
Expand Down Expand Up @@ -181,6 +181,3 @@ def test_get_reference_properties(self):
def test_get_qualifier_properties(self):
print(self.common_item.get_qualifier_properties(prop_id='P170'))
self.assertTrue(len(self.common_item.get_qualifier_properties(prop_id='P2067')))

def test_get_linked_by(self):
self.assertTrue(len(wbi_core.FunctionsEngine.get_linked_by('Q2')))
4 changes: 2 additions & 2 deletions test/test_wbi_login.py
Expand Up @@ -5,7 +5,7 @@

import pytest

from wikibaseintegrator import wbi_login, wbi_core
from wikibaseintegrator import wbi_login, wbi_functions

# look for environment variables. if none set, don't do anything
WDUSER = os.getenv("WDUSER")
Expand All @@ -23,4 +23,4 @@ def test_write():
if WDUSER and WDPASS:
login = wbi_login.Login(WDUSER, WDPASS)
with pytest.raises(ValueError):
wbi_core.FunctionsEngine.mediawiki_api_call_helper(data=None, login=login, mediawiki_api_url='https://unsdfdskfjljzkerezr.org/w/api.php')
wbi_functions.mediawiki_api_call_helper(data=None, login=login, mediawiki_api_url='https://unsdfdskfjljzkerezr.org/w/api.php')