Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Menu
Open sidebar
Institut für Informatik
dbis
software
dbispipeline
Commits
16e42bb5
Commit
16e42bb5
authored
Dec 23, 2021
by
Benjamin Murauer
Browse files
release r0.8.30
parent
4a146315
Pipeline
#72664
passed with stage
in 2 minutes and 30 seconds
Changes
3
Pipelines
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
5 additions
and
3 deletions
+5
-3
setup.cfg
setup.cfg
+1
-1
setup.py
setup.py
+1
-1
src/dbispipeline/dataloaders/explicit.py
src/dbispipeline/dataloaders/explicit.py
+3
-1
No files found.
setup.cfg
View file @
16e42bb5
[bumpversion]
current_version = 0.8.
29
current_version = 0.8.
30
[bumpversion:file:setup.py]
...
...
setup.py
View file @
16e42bb5
...
...
@@ -11,7 +11,7 @@ with open('README.md') as fh:
setup
(
name
=
'dbispipeline'
,
version
=
'0.8.
29
'
,
version
=
'0.8.
30
'
,
author
=
'Benjamin Murauer, Michael Vötter'
,
author_email
=
'b.murauer@posteo.de'
,
description
=
'should make things more reproducible'
,
...
...
src/dbispipeline/dataloaders/explicit.py
View file @
16e42bb5
...
...
@@ -31,8 +31,9 @@ def _limit(
)
->
Tuple
[
pd
.
DataFrame
,
np
.
array
]:
df
,
key
=
_attach
(
dataset_part
[
0
],
dataset_part
[
1
])
sub_df
=
df
[
df
[
key
].
isin
(
remaining_targets
)]
first_column
=
sub_df
.
columns
[
0
]
groups
=
sub_df
.
groupby
(
key
)
min_population
=
groups
.
count
()[
'text_raw'
].
min
()
min_population
=
groups
.
count
()[
first_column
].
min
()
if
max_docs_per_target
:
if
min_population
>
max_docs_per_target
:
sub_df
=
sub_df
.
groupby
(
key
).
sample
(
max_docs_per_target
)
...
...
@@ -178,6 +179,7 @@ class TrainTestSplitLoader(Loader):
self
.
max_targets
,
len
(
all_targets
),
)
selected_targets
=
list
(
all_targets
)
else
:
selected_targets
=
list
(
all_targets
)
train
=
_limit
(
train
,
selected_targets
,
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment