Textual Data Analysis Package used by the TXM Software

Description

Statistical exploration of textual corpora using several methods from French 'Textometrie' (new name of 'Lexicometrie') and French 'Data Analysis' schools. It includes methods for exploring irregularity of distribution of lexicon features across text sets or parts of texts (Specificity analysis); multi-dimensional exploration (Factorial analysis), etc. Those methods are used in the TXM software.

Details

Index:

specificities    Compute Lexical Specificity of subcorpus 
progression    Draw progression graphic 

Author(s)

Sylvain Loiseau, Lise Vaudor, Matthieu Decorde, Lise Vaudor

Examples

data(robespierre);
specificities(robespierre);

adverbs frequency from 5 different domains of the BFM database

Description

A lexical table containing frequencies of adverbs from the BFM (Base de Francais m\'edi\'eval) database in 5 different domains (literary, historical, didactic, law, religious).

Usage

data(bfm)

Format

The format is: num [1:2, 1:5] 103000 1370887 23429 413441 15345 ... - attr(*, "dimnames")=List of 2 ..$ : chr [1:2] "ADV" "other" ..$ : chr [1:5] "literary" "history" "didactic" "juridical" ...

Details

The last line of the table gives the total frequency of all the other part of speech words in each of these domains.

Source

BFM: https://bfm.ens-lyon.fr

References

BFM - Base de Fran\,cais M\'edi\'eval [En ligne]. Lyon : ENS de Lyon, Laboratoire ICAR, 2012, https://bfm.ens-lyon.fr.

Draw progression graphic

Description

Draw the progression graphic of matches of CQL queries in a corpus

Usage

  progression(positions, names, colors, styles, widths, corpusname, Xmin, T, 
  	doCumulative, structurepositions, strutnames, graphtitle, bande)

Arguments

Author(s)

Matthieu Decorde

5 words from Robespierre's discourses

Description

A lexical table containing frequencies of 5 words from 9 different public discourses of French politician Robespierre (between november 1793 and july 1794).

Usage

data(robespierre)

Format

The format is: num [1:6, 1:10] 464 45 35 30 6 ... - attr(*, "dimnames")=List of 2 ..$ : chr [1:6] "de" "peuple" "republique" "ennemi" ... ..$ : chr [1:10] "D1" "D2" "D3" "D4" ...

Details

The last line of the table gives the total frequency of all the other forms in each of these discourses.

Source

Lafon P. (1980) Sur la variabilit\'e de la fr\'e quence des formes dans un corpus, Mots, 1, pp. 127–165.

References

Lafon P. (1980) Sur la variabilit\'e de la fr\'e quence des formes dans un corpus, Mots, 1, pp. 127–165.

Examples

data(robespierre)

## See graphic in Lafon, 1980 - page 140

t <- colSums(robespierre)["D9"];     # size of the part
T <- sum(robespierre);               # size of the corpus
f <- rowSums(robespierre)["peuple"]; # total frequency of "peuple"
p <- dhyper(1:15, f, T-f, t)
title <- "Probability of each frequency of 'peuple' in the 'D9' discourse from 1 to 15"
plot(p, type="h", main=title, xlab="k", ylab="Prob(k)");

Calculate Lexical Specificity Score

Description

Calculate the specificity - or association or surprise - score of a word being present f times or more in a sub-corpus of t words given that it appears a total of F times in a whole corpus of T words.

Usage

specificities(lexicaltable, types=NULL, parts=NULL)

Arguments

Value

Returns a matrix of nrow(lexicaltable) * ncol(lexicaltable) (the number of rows and columns may be reduced using types or parts), each cell giving the specificity score.

Author(s)

Matthieu Decorde, Serge Heiden, Sylvain Loiseau, Lise Vaudor

References

Lafon P. (1980) Sur la variabilit\'e de la fr\'e quence des formes dans un corpus, Mots, 1, pp. 127–165. https://www.persee.fr/doc/mots_0243-6450_1980_num_1_1_1008

See Also

specificities.probabilities, specificities.lexicon

Examples

data(robespierre);
spe <- specificities(robespierre);
string <- paste("The word %s appears f=%d times in a sub-corpus of t=%d words,",
" given a total frequency of F=%d in the robespierre corpus made",
" of T=%d words. The corresponding specificity score is %f", sep="");
print(sprintf(string,
'peuple',
robespierre['peuple','D4'],
colSums(robespierre)['D4'],
rowSums(robespierre)['peuple'],
sum(robespierre),
spe['peuple', 'D4']));

Display specificities probability

Description

Display specificities probability distribution (call dhyper and specificities.probabilities.vector)

Usage

  specificities.distribution.plot(x, F, t, T)

Arguments

Value

nothing

Author(s)

Matthieu Decorde, Serge Heiden

*OBSOLETE FUNCTION (see 'specificities.lexicon.new')* specificities association score with two frequency lists.

Description

Compute specificities association score between a lexicon and a sub-lexicon

Usage

specificities.lexicon(lexicon, sublexicon)

Arguments

Value

specificities index as a named vector.

See Also

specificities for specificities score and references

specificities association score with two frequency list.

Description

Compute specificities association score between a lexicon and a sub-lexicon. A new version of the "specificities.lexicon" function

Usage

specificities.lexicon.new(lexicon, sublexicon)

Arguments

Value

specificities index as a named vector.

See Also

specificities for specificities score and references

Calculate specificity probabilities

Description

Utility function computing specificity probabilities for the specificities function.

Usage

specificities.probabilities(lexicaltable, types = NULL, parts = NULL)

Arguments

Value

Returns a matrix of signed specificity probabilities (between -1.0 and 1.0). By convention:

See Also

see specificities.

Vector raw hypergeometric probabilities

Description

Calculate specificity probabilities on vector (call phyper and phyper_right)

Usage

  specificities.probabilities.vector(v_f, v_F, T, t)

Arguments

Value

Hypergeometric probabilities. See specificities.lexicon.

Author(s)

Matthieu Decorde, Serge Heiden