Index: languages/apertium-srd/apertium-srd.srd.dix
===================================================================
--- languages/apertium-srd/apertium-srd.srd.dix (revision 70097)
+++ languages/apertium-srd/apertium-srd.srd.dix (revision 70101)
@@ -6275,6 +6275,7 @@
ZaponeGiappone
JiaponiGiappone
+ Islanda
Ispagna
SpànniaIspagna
IspannaIspagna
@@ -6284,7 +6285,7 @@
GermaniaGermània
ZermàniaGermània
Cuba
- Canada
+ Canada
CànadaCanada
Austria
ÀustriaAustria
@@ -6389,7 +6390,6 @@
Turri
ÙssaraManna
Biddexidru
- BiddaMara
BiddanoaForru
BiddanoaFranca
@@ -12694,7 +12694,6 @@
PS
Queen
Rai
- RAI
Ralph
Ray
Raymond
@@ -12765,7 +12764,6 @@
UFC
UK
Unesco
- UNESCO
Universal
URSS
US
@@ -51042,6 +51040,7 @@
VillacidroBiddexidru
BiddaMara
VillamarBiddaMara
+ BissaMaraBiddaMara
VillanovaforruBiddanoaForru
VillanovafrancaBiddanoaFranca
Azos
Index: languages/apertium-ita/apertium-ita.ita.dix
===================================================================
--- languages/apertium-ita/apertium-ita.ita.dix (revision 70097)
+++ languages/apertium-ita/apertium-ita.ita.dix (revision 70101)
@@ -13609,7 +13609,6 @@
Thomas
Afghanistan
ABC
- FIAT
Abad
Abel
Adam
@@ -14816,7 +14815,6 @@
Carminuccio
Giuliano
Amburgo
- Canada
Venera
Landa
Etna
@@ -15846,7 +15844,6 @@
Oschiri
Padru
LuPalau
- Sant'AntoniodiGallura
SantaTeresadiGallura
SanTeodoro
Telti
@@ -43523,6 +43520,7 @@
PatruPadru
LuPalauPalau
Sant'AntoniodiGallura
+ Sant'AntoniodiGalluraSant'AntoniodiGallura
SantuAntoni'eCalanzanusSant'AntoniodiGallura
Sant'AntoniSant'AntoniodiGallura
SantaTeresaGallura
@@ -44516,7 +44514,6 @@
raffigurant
raffigurat
Rai
- RAI
Ralph
ranking
rap
@@ -44660,7 +44657,6 @@
UFC
UK
Unesco
- UNESCO
Universal
universitari
universitari
Index: incubator/apertium-srd-ita/apertium-srd-ita.srd-ita.dix
===================================================================
--- incubator/apertium-srd-ita/apertium-srd-ita.srd-ita.dix (revision 70097)
+++ incubator/apertium-srd-ita/apertium-srd-ita.srd-ita.dix (revision 70101)
@@ -1773,7 +1773,7 @@
mancu
presente
cale
- da
+ de
pre
anti
pro
@@ -5804,7 +5804,7 @@
Euskadi
Filadèlfia
Firenze
- Giapone
+ Giappone
Girona
Guyana
Hanoi
@@ -5854,7 +5854,6 @@
Sàrdara
Segariu
Serramanna
- SanGavinoMonreale
Serramanna
Setzu
Siddi
@@ -5862,56 +5861,372 @@
Turri
ÙssaraManna
Biddexidru
- BissaMara
BiddanoaForru
BiddanoaFranca
- Azos
- Agghju
+ Abbasanta
Aglientu
- SantuFranciscud'Aglièntu
- Alà
- Alzachena
- Altzaghena
- Badesi
- Belchidda
- Bilchidda
- Bortigiadas
- Bultigghjata
+ Alà
+ Àldara
+ Ales
+ Àllai
+ Altzaghena
+ Anela
+ Arborea
+ Arbus
+ Arcidanu
+ Ardaule
+ Aristanis
+ Aritzu
+ Armungia
+ Arriora
+ Arruinas
+ Àrthana
+ Assèmini
+ Assou
+ Asuni
+ Atzara
+ Austis
+ Azos
+ Badesi
+ Bàini
+ Ballau
+ Bànari
+ Baradili
+ Baressa
+ Barì
+ Barrali
+ Barùmini
+ Baulau
+ Baunei
+ Belchidda
+ Benetutti
+ Bessude
+ BiddadePutzu
+ BiddadeSartu
+ BiddaMannaIstrisàili
+ BiddaMara
+ BiddaSorris
+ BiddaSpetziosa
+ Biddamatraxa
+ BiddanoaMonteleone
+ BiddanoaTruschedu
+ BiddanoaTulu
+ Biddaramosa
+ Biddaubrana
+ Bidumajore
+ Biduniu
+ Bilartzi
+ Bìroro
+ Boàtiri
+ Bolòtana
+ Bonàrcadu
+ Bonolva
+ Bono
+ Boroneddu
+ Bòrore
+ Bortigale
+ Bortigiadas
+ Boruta
+ Bosa
+ Bottidda
+ Brevìe
Buddusò
- Budùne
- Budòni
- Calanzanus
- Calagnani
+ Budùne
+ Buggerru
+ Bultei
+ Bultzi
+ Bunnànnaru
+ Burcei
+ Busache
+ Cabuderra
+ CalaSeda
+ Calanzanus
+ Cardedu
+ Carluforti
+ Carzeghe
+ CastedduSardu
+ Casteddu
+ Castiadas
+ Ceraxus
+ Cherèmule
+ Codaruina
+ Codronzanu
+ Conesa
+ Cossoine
+ Crabas
+ Crabonaxa
+ Crabònia
+ Crucuris
+ CuartuSant'Aleni
+ Cuartuciu
+ Cùllieri
+ Cuzinas
+ Dèsulu
+ DèximuMannu
+ DèximuPutzu
+ Domunoas
+ DomusdeMaria
+ Donòri
+ Duarche
+ Durgali
+ Elini
+ Èrula
Figari
- SaMadalena
- AMadalena
- Loèri-PoltuSantuPaulu
- Lòiri-PoltuSantuPaulu
+ Fiolinas
+ Flùssio
+ Foghesu
+ Fonne
+ Fordongianus
+ Forru
+ FrùminiMayori
+ Furtei
+ Gadoni
+ Gàiru
+ Galtellì
+ Gavoi
+ Genoni
+ Genuri
+ Gèsigu
+ Gèsturi
+ Giave
+ Gibba
+ Goni
+ GonnosFanàdiga
+ Gonnoscodina
+ Gonnosnò
+ Gonnostramatza
+ Guamajori
+ Guasila
+ Gùspini
+ Iersu
+ Igrèsias
+ Illorai
+ Irbonu
+ Irgoli
+ IsPratzas
+ Iscanu
+ Ìsili
+ Isporalatu
+ Istertzili
+ Istintinu
+ Itireddu
+ ÌtiriCannedu
+ Jelisuli
+ Làcana
+ Laerru
+ Lanusèi
+ Lei
+ Loceri
+ Loculi
+ Lodè
+ Lodine
Logusantu
- Locusantu
- Lùrisi
- Luras
- Monti
- Monte
- Terranoa
- Tarranoa
- Òscari
- Oscheri
- Oscheri
+ Lotzorai
+ Lungone
+ Luras
+ Lùvula
+ Macumere
+ Magumadas
+ Maltis
+ Mamujada
+ Mandas
+ Maracalagonis
+ Mara
+ Marrùbiu
+ Masainas
+ Masuddas
+ Meana
+ Miris
+ Mòdolo
+ Mogorella
+ Mòguru
+ Monteleone
+ Monte
+ Montresta
+ Mores
+ Mragaxori
+ Murera
+ Muristeni
+ Muros
+ Musei
+ Narabuia
+ Narcau
+ Neunele
+ Noragugume
+ Norghiddu
+ NugheduSantuNigola
+ Nughedu
+ Nùgoro
+ Nule
+ Nulvi
+ Nurachi
+ Nuradda
+ Nuragus
+ Nuràminis
+ Nureci
+ Nurri
+ Nuxis
+ Ollasta
+ Ollolai
+ Onanie
+ Oniai
+ Onieri
+ Orane
+ Orgòsolo
+ Orosei
+ Oroteddi
+ Orroli
+ Ortacesus
+ Orthullè
+ Ortueri
+ Ortzai
+ Orune
+ Oscheri
+ Osidde
+ Osilo
+ Osini
+ Ossi
+ Othieri
+ Otzana
+ Ovodda
+ Pabillonis
+ Pàdria
Padru
- LuPalau
- SantuAntonideCalanzanus
- SantuAntoni
- Lungone
- Lungoni
- SantuTiadoru
+ Palau
+ Partiolla
+ Patada
+ Paule
+ PauliArbarei
+ PauliGerrei
+ Pauli
+ Pau
+ Pèifugas
+ Perdaxus
+ Piaghe
+ Piscinas
+ Pompu
+ Portescusi
+ PortuTurre
+ Portuscusi
+ Potuvigari
+ Pramas
+ Pramentelu
+ Pula
+ Putumajore
+ Rumana
+ SaBaronia
+ SaMadalena
+ Sàdili
+ Sàgama
+ S'Alighera
+ Samassi
+ Samatzai
+ Samugheu
+ SantaJusta
+ Santadi
+ Sant'AnnaArresi
+ Sant'Eru
+ Santu'Ainju
+ SantuAndriaFrius
+ SantuAntiogu
+ SantuAntoniArruinas
+ SantuAntoni'eCalanzanus
+ Santu'Asili
SantuDeadoru
- Telti
- Teltis
+ Santu'Idu
+ SantuLussurzu
+ SantuPauluLoèri
+ SantuPedru
+ SantuSperau
+ Santu'UanniSruexu
+ Sarrocu
+ Sarule
+ Scalepranu
+ Scroca
+ Séddini
+ Seddori
+ Sèdilo
+ Sèlegas
+ Semèstene
+ Sèneghe
+ Senis
+ Sennariolo
+ Sènnaru
+ Senorbì
+ Serdiana
+ Serrenti
+ Serri
+ Sètimu
+ Seui
+ Seulu
+ Siamanna
+ Siddi
+ Siimajori
+ Siipicia
+ Silanos
+ Silìcua
+ Siligo
+ Silius
+ Simaghis
+ Sìmala
+ Sindia
+ Sini
+ Sìnnia
+ Siris
+ Soddie
+ Solarussa
+ Solèminis
+ Sòrgunu
+ Sorradile
+ Sòssu
+ SriugusDonigalla
+ SuBurgu
+ SuMasu
+ Sueddi
+ S'Ulumedu
+ Sune
+ Tadasune
+ Talana
+ Tàtari
+ Teltis
Tèmpiu
- Trinidade
- TrinitaieVignola
+ Tergu
+ Terranoa
+ Terrarba
+ Tertenia
+ Teti
+ Thiesi
+ Tìana
+ Tinnura
+ Tissi
+ Tonara
+ Torpè
+ Tortuelie
+ Tratalias
+ Tresnuraghes
+ Triè
+ TrinitaieVignolaTrinidade
+ Tuili
+ Tula
+ Turalva
+ Turri
+ Tzaramonte
+ Tzeddiani
+ Tzorfuliu
+ Uda
+ Ula
+ Ulassa
+ Ulìana
+ Uras
+ Uri
+ Useddus
+ Usini
+ Ùssana
+ Ussassa
+ Viddalba
+ Vitzi
+ Xrexei
Alighèra
@@ -13135,361 +13450,6 @@
Palermu
Casteddu
- Arbus
- Barùmini
- Forru
- Furtei
- Genuri
- Gèsturi
- GonnosFanàdiga
- Gùspini
- IsPratzas
- Pabillonis
- PauliArbarei
- Samassi
- Santu'Ainju
- Seddori
- Serrenti
- Siddi
- Tuili
- Turri
- BiddaMara
- Azos
- Alà
- Altzaghena
- Badesi
- Belchidda
- Bortigiadas
- Budùne
- Calanzanus
- SaMadalena
- SantuPauluLoèri
- Luras
- Monte
- Oscheri
- Palau
- SantuAntoni'eCalanzanus
- Lungone
- Teltis
- TrinitaieVignolaTrinidade
- Abbasanta
- Bidumajore
- Ales
- Àllai
- Arborea
- Ardaule
- Assou
- Asuni
- Baradili
- Boàtiri
- Baressa
- Baulau
- Biduniu
- Bonàrcadu
- Boroneddu
- Bosa
- Busache
- Crabas
- Cùllieri
- Crucuris
- Flùssio
- Fordongianus
- Genoni
- Bilartzi
- Gonnoscodina
- Gonnosnò
- Gonnostramatza
- Làcana
- Magumadas
- Marrùbiu
- Masuddas
- Miris
- Mòdolo
- Mogorella
- Mòguru
- Montresta
- Mragaxori
- Narabuia
- Neunele
- Norghiddu
- Nughedu
- Nurachi
- Nureci
- Ollasta
- Aristanis
- Pramas
- Pau
- Paule
- Pompu
- Arriora
- Arruinas
- Sàgama
- Samugheu
- Arcidanu
- SantaJusta
- SantuLussurzu
- Sant'Eru
- Iscanu
- Sèdilo
- Sèneghe
- Senis
- Sennariolo
- Siimajori
- Siamanna
- Siipicia
- Sìmala
- Simaghis
- Sini
- Siris
- Soddie
- Solarussa
- Sorradile
- Sune
- Tadasune
- Terrarba
- Tinnura
- Tresnuraghes
- Ula
- Uras
- Useddus
- BiddanoaTruschedu
- SantuAntoniArruinas
- Biddaubrana
- Bàini
- Tzeddiani
- Tzorfuliu
- Aritzu
- Atzara
- Austis
- Brevìe
- Bìroro
- Vitzi
- Bolòtana
- Bòrore
- Bortigale
- Dèsulu
- Durgali
- Duarche
- Fonne
- Gadoni
- Galtellì
- Gavoi
- Irgoli
- Lei
- Loculi
- Lodè
- Lodine
- Lùvula
- Macumere
- Mamujada
- Meana
- Noragugume
- Nùgoro
- Ulìana
- Ollolai
- Ortzai
- Onanie
- Oniai
- Onieri
- Orane
- Orgòsolo
- Orosei
- Oroteddi
- Ortueri
- Orune
- Osidde
- Otzana
- Ovodda
- Sarule
- Silanos
- Sindia
- Sòrgunu
- Teti
- Tìana
- Tonara
- Torpè
- S'Alighera
- Anela
- Àldara
- Bànari
- Benetutti
- Bessude
- Bunnànnaru
- Bono
- Bonolva
- Boruta
- Bottidda
- Bultei
- Bultzi
- SuBurgu
- Carzeghe
- CastedduSardu
- Cherèmule
- Tzaramonte
- Codronzanu
- Cossoine
- Èrula
- Isporalatu
- Fiolinas
- Giave
- Illorai
- Itireddu
- ÌtiriCannedu
- Laerru
- Mara
- Maltis
- Monteleone
- Mores
- Muros
- NugheduSantuNigola
- Nule
- Nulvi
- S'Ulumedu
- Osilo
- Ossi
- Othieri
- Pàdria
- Patada
- Pèifugas
- Piaghe
- PortuTurre
- Putumajore
- Potuvigari
- Rumana
- Cuzinas
- Tàtari
- Séddini
- Semèstene
- Sènnaru
- Siligo
- Sòssu
- Istintinu
- Tergu
- Thiesi
- Tissi
- Turalva
- Tula
- Uri
- Usini
- Codaruina
- Viddalba
- BiddanoaMonteleone
- Àrthana
- Barì
- Baunei
- Cardedu
- Elini
- Gàiru
- Jelisuli
- Irbonu
- Iersu
- Lanusèi
- Loceri
- Lotzorai
- Osini
- Foghesu
- Seui
- Talana
- Tertenia
- Tortuelie
- Triè
- Ulassa
- Orthullè
- Ussassa
- BiddaMannaIstrisàili
- Buggerru
- CalaSeda
- Crabònia
- Carluforti
- Domunoas
- FrùminiMayori
- Gibba
- Conesa
- Igrèsias
- Masainas
- Musei
- Narcau
- Nuxis
- Perdaxus
- Piscinas
- Portescusi
- Portuscusi
- Santu'UanniSruexu
- Santadi
- Sant'AnnaArresi
- SantuAntiogu
- Tratalias
- Biddamatraxa
- SaBaronia
- Armungia
- Assèmini
- Ballau
- Barrali
- Burcei
- Casteddu
- Cabuderra
- Castiadas
- DèximuMannu
- DèximuPutzu
- Partiolla
- DomusdeMaria
- Donòri
- SuMasu
- Scalepranu
- Scroca
- Istertzili
- Xrexei
- Gèsigu
- Goni
- Guamajori
- Guasila
- Ìsili
- Mandas
- Maracalagonis
- Muristeni
- Pauli
- Murera
- Nuragus
- Nuradda
- Nuràminis
- Nurri
- Orroli
- Ortacesus
- Pramentelu
- Pula
- Cuartuciu
- CuartuSant'Aleni
- Sàdili
- Samatzai
- Santu'Asili
- PauliGerrei
- SantuSperau
- SantuAndriaFrius
- Santu'Idu
- Sarrocu
- Ceraxus
- Sèlegas
- Senorbì
- Serdiana
- Serri
- Sètimu
- Seulu
- Silìcua
- Silius
- Sìnnia
- SriugusDonigalla
- Solèminis
- Sueddi
- Ùssana
- Uda
- Biddaramosa
- BiddanoaTulu
- BiddadePutzu
- BiddadeSartu
- SantuPedru
- Crabonaxa
- BiddaSorris
- BiddaSpetziosa
Sir
Lord
@@ -13847,7 +13807,6 @@
PS
Queen
Rai
- RAI
Ralph
Raymond
Raymond
@@ -13928,7 +13887,6 @@
UFC
UK
Unesco
- UNESCO
Universal
URSS
US
@@ -13986,7 +13944,6 @@
Caballé
Cadillac
Califòrnia
- Canada
Cannes
Cardiff
Carìntzia
Index: incubator/apertium-srd-ita/dev/testvoc/generation.sh
===================================================================
--- incubator/apertium-srd-ita/dev/testvoc/generation.sh (revision 70097)
+++ incubator/apertium-srd-ita/dev/testvoc/generation.sh (revision 70101)
@@ -2,28 +2,6 @@
set -e -u
-# You can override the below variables by doing e.g.
-# $ export CYCLES=1 BLOCK=1M
-# before running this script.
-
-# How many times to follow cycle when expanding with --hfst; gets slow if too high:
-declare -ir CYCLES=${CYCLES-0}
-# How many parallel pipelines to run (requires GNU parallel installed;
-# only worth increasing if CPU's are not saturated and there's free
-# RAM while running):
-declare -ir J=${J-1}
-# How much data to translate before restarting the pipeline (some
-# pipelines have memory leaks and need restarting every so often):
-declare -r BLOCK=${BLOCK:-100M}
-
-
-if [[ $# -ge 1 && $1 = --hfst ]]; then
- HFST=true
- shift
-else
- HFST=false
-fi
-
if [[ $# -eq 1 ]]; then
mode=$1
dix=guess
@@ -34,63 +12,27 @@
cat >&2 <]:' '
- /:<:/ {next}
- $2 ~ /|DUE_TO_LT_PROC_HANG|__REGEXP__/ {next}
- {
- esc=$2
- gsub("/","\\/",esc)
- gsub("^","\\^",esc)
- gsub("$","\\$",esc)
- print "["esc"] ^"$1"/"$2"$ ^./."clb"$"
- }'
-}
-
-analyser_to_hfst () {
- case "$(head -c4 "$1")" in
- HFST)
- hfst-fst2fst -t "$1"
- ;;
- *) # lttoolbox bin's start with their 's :(
- lt-print "$1" \
+ lt-print "$1".automorf.bin \
| sed 's/ /@_SPACE_@/g' \
- | hfst-txt2fst -e ε
- ;;
- esac
-}
-
-analysis_expansion_hfst () {
- analyser_to_hfst "$1" \
+ | hfst-txt2fst -e ε \
| hfst-project -p lower \
- | hfst-fst2strings -c"${CYCLES}" \
+ | hfst-fst2strings -c0 \
| awk -v clb="$2" '
/[][$^{}\\]/{next} # skip escaping hell
- /|DUE_TO_LT_PROC_HANG|__REGEXP__/ {next}
{
gsub("]","\\]")
esc=$0
@@ -102,105 +44,55 @@
# give the "disambiguated" output, no forms
}
-only_errs () {
- if [[ $# -ge 1 && $1 = --no-@ ]]; then
- atfilter () { grep -v '].*/@'; }
+split_ambig () {
+ if command -V pypy3 &>/dev/null; then
+ python=pypy3
else
- atfilter () { cat; }
+ python=python3
fi
- # turn escaped SOLIDUS into DIVISION SLASH, so we don't grep correct stuff ("A/S" is a possible lemma)
- sed 's%\\/%∕%g' |\
- atfilter |\
- grep '][^<]*[#/]'
+ PYTHONPATH="$(dirname "$0"):${PYTHONPATH:-}" "${python}" -c '
+from streamparser import parse_file, readingToString
+import sys
+for blank, lu in parse_file(sys.stdin, withText=True):
+ print(blank+" ".join("^{}/{}$".format(lu.wordform, readingToString(r))
+ for r in lu.readings),
+ end="")'
+
}
-run_mode () {
- if command -V parallel &>/dev/null; then
- parallel -j"$J" --pipe --block "${BLOCK}" -- bash "$@"
- else
- bash "$@"
- fi
+mode_after_analysis ()
+{
+ eval $(grep '|' "$1" |\
+ sed 's/[^|]*|//' |\
+ sed 's/.*apertium-pretransfer/apertium-pretransfer/' |\
+ sed 's/lt-proc -p[^|]*/cat/' |\
+ sed 's/autobil.bin *|/& split_ambig |/' |\
+ sed 's/\$1/-d/g;s/\$2//g')
+ # lt-proc -p fails
}
-declare -a TMPFILES
-cleanup () {
- for f in "${TMPFILES[@]}"; do
- rm -f "$f"
- done
+only_errs () {
+ grep '][^<]*[#/]'
}
-trap 'cleanup' EXIT
-PYTHONPATH="$(dirname "$0"):${PYTHONPATH:-}"
-export PYTHONPATH
-if command -V pypy3 &>/dev/null; then
- python=pypy3
-else
- python=python3
+lang1=${mode%%-*}
+
+if [[ ${dix} = guess ]]; then
+ lang1dir=$(grep -m1 "^AP_SRC.*apertium-${lang1}" config.log | sed "s/^[^=]*='//;s/'$//")
+ dix=${lang1dir}/apertium-${lang1}.${lang1}.dix
fi
-split_ambig=$(mktemp -t gentestvoc.XXXXXXXXXXX)
-TMPFILES+=("${split_ambig}")
-cat >"${split_ambig}" < "${mode_after_analysis}"
-
-mode_after_tagger=$(mktemp -t gentestvoc.XXXXXXXXXXX)
-TMPFILES+=("${mode_after_tagger}")
-grep '|' modes/"${mode}".mode \
- | sed 's/[^|]*|//' \
- | sed 's/.*apertium-pretransfer/apertium-pretransfer/' \
- | sed 's/lt-proc -p[^|]*/cat/' \
- | sed "s%autobil.bin'* *|%& ${split_ambig} |%" \
- | sed 's/\$1/-d/g;s/\$2//g' \
- > "${mode_after_tagger}"
-# lt-proc -p fails, that's why we remove that
-
-
-lang1=${mode%%-*}
-
clb=""
case ${lang1} in
nno|nob) clb="" ;;
esac
-if $HFST; then
- if [[ ${dix} = guess ]]; then
- dix=$(xmllint --xpath "string(/modes/mode[@name = '${mode}']/pipeline/program[1]/file[1]/@name)" modes.xml)
- fi
- analysis_expansion_hfst "${dix}" "${clb}" \
- | run_mode "${mode_after_tagger}" \
+# Make it possible to edit the .dix while testvoc is running:
+dixtmp=$(mktemp -t gentestvoc.XXXXXXXXXXX)
+trap 'rm -f "${dixtmp}"' EXIT
+cat "${dix}" > "${dixtmp}"
+
+analysis_expansion "${mode}" "${clb}" \
+ | mode_after_analysis modes/"${mode}".mode \
| only_errs
-else
- if [[ ${dix} = guess ]]; then
- lang1dir=$(grep -m1 "^AP_SRC.*apertium-${lang1}" config.log | sed "s/^[^=]*='//;s/'$//")
- dix=${lang1dir}/apertium-${lang1}.${lang1}.dix
- fi
- # Make it possible to edit the .dix while testvoc is running:
- dixtmp=$(mktemp -t gentestvoc.XXXXXXXXXXX)
- TMPFILES+=("${dixtmp}")
- cat "${dix}" > "${dixtmp}"
- analysis_expansion "${dixtmp}" "${clb}" \
- | run_mode "${mode_after_analysis}" \
- | only_errs --no-@
-fi
Index: incubator/apertium-srd-ita/dev/testvoc/streamparser.py
===================================================================
--- incubator/apertium-srd-ita/dev/testvoc/streamparser.py (revision 70097)
+++ incubator/apertium-srd-ita/dev/testvoc/streamparser.py (revision 70101)
@@ -9,36 +9,7 @@
import re, pprint, sys, itertools, fileinput
from collections import namedtuple
-
-class Knownness:
- __doc__ = """Level of knowledge associated with a lexical unit.
- Values:
- known
- unknown: Denoted by '*', analysis not available.
- biunknown: Denoted by '@', translation not available.
- genunknown: Denoted by '#', generated form not available.
-"""
- symbol = ""
-
-class known(Knownness):
- pass
-class unknown(Knownness):
- symbol = "*"
-class biunknown(Knownness):
- symbol = "@"
-class genunknown(Knownness):
- symbol = "#"
-
SReading = namedtuple('SReading', ['baseform', 'tags'])
-try:
- SReading.__doc__ = """A single subreading of an analysis of a token.
- Fields:
- baseform (str): The base form (lemma, lexical form, citation form) of the reading.
- tags (list of str): The morphological tags associated with the reading.
-"""
-except AttributeError:
- # Python 3.2 users have to read the source
- pass
def subreadingToString(sub):
return sub.baseform+"".join("<"+t+">" for t in sub.tags)
@@ -68,10 +39,8 @@
lexicalUnit (str): The lexical unit in Apertium stream format.
wordform (str): The word form (surface form) of the lexical unit.
readings (list of list of SReading): The analyses of the lexical unit with sublists containing all subreadings.
- knownness (Knownness): The level of knowledge of the lexical unit.
"""
- knownness = known
def __init__(self, lexicalUnit):
self.lexicalUnit = lexicalUnit
@@ -82,7 +51,8 @@
self.readings = []
for reading in readings:
if len(reading) < 1:
- print("WARNING: Empty readings for {}".format(self.lexicalUnit), file=sys.stderr)
+ #print("WARNING: Empty readings for {}".format(self.lexicalUnit), file=sys.stderr)
+ pass
elif reading[0] not in '*#@':
subreadings = []
@@ -94,8 +64,6 @@
subreadings.append(SReading(baseform=baseform, tags=tags))
self.readings.append(subreadings)
- else:
- self.knownness = {'*': unknown, '@': biunknown, '#': genunknown}[readings[0][0]]
def __repr__(self):
return self.lexicalUnit