Example usage for edu.stanford.nlp.stats Distribution getDistribution

Introduction

In this page you can find the example usage for edu.stanford.nlp.stats Distribution getDistribution.

Prototype

public static <E> Distribution<E> getDistribution(Counter<E> counter) 

Source Link

Document

Creates a Distribution from the given counter.

Usage

From source file:nate.reading.SlotInducer.java

private boolean argVectorsDiffer(Counter<String> args1, Counter<String> args2) {
    System.out.println("argVectorsDiffer top!");
    Distribution<String> dist1 = Distribution.getDistribution(args1);
    Distribution<String> dist2 = Distribution.getDistribution(args2);

    Set<String> argdiffs = new HashSet<String>();

    for (String token : dist1.keySet()) {
        double prob1 = dist1.getCount(token);
        if (dist1.getCount(token) > 0.02) {
            double prob2 = dist2.getCount(token);
            double ratio = (prob1 < prob2 ? prob1 / prob2 : prob2 / prob1);
            System.out.printf("- %s\t%.4f\t%.4f\tratio=%.4f\n", token, prob1, prob2, ratio);
            if (ratio < 0.2) {
                argdiffs.add(token);//from   ww w.  ja va  2s .c o m
                System.out.println("  arg differs: " + token);
            }
        }
    }

    if (argdiffs.size() >= 2) {
        System.out.println("Arg vectors differ!!");
        return true;
    }

    return false;
}

---