SAS Functions: SOUNDEX, COMPGED, and Their Alternatives

Introduction

In SAS, the SOUNDEX and COMPGED functions are powerful tools for text comparison, particularly when dealing with names or textual data that may have variations. In addition to these, SAS offers other functions like DIFFERENCE and SPEDIS that provide additional ways to measure similarity and distance between strings. This article explores these functions, provides examples, and compares their uses.

The SOUNDEX Function

The SOUNDEX function converts a character string into a phonetic code. This helps in matching names that sound similar but may be spelled differently. The function generates a four-character code based on pronunciation.

Syntax

SOUNDEX(string)

Where string is the character string you want to encode.

Example

data names;
    input name $20.;
    soundex_code = soundex(name);
    datalines;
John
Jon
Smith
Smythe
;
run;

proc print data=names;
run;

In this example, "John" and "Jon" have the same SOUNDEX code, reflecting their similar pronunciation, while "Smith" and "Smythe" have different codes.

The COMPGED Function

The COMPGED function measures the similarity between two strings using the Generalized Edit Distance algorithm. This function is useful for fuzzy matching, especially when dealing with misspelled or slightly varied text.

Syntax

COMPGED(string1, string2)

Where string1 and string2 are the strings to compare.

Example

data comparisons;
    string1 = 'John';
    string2 = 'Jon';
    distance = compged(string1, string2);
    datalines;
;
run;

proc print data=comparisons;
run;

The COMPGED function returns a numerical value representing the edit distance between "John" and "Jon". Lower values indicate higher similarity.

Alternative Functions

The DIFFERENCE Function

The DIFFERENCE function returns the difference between the SOUNDEX values of two strings. This function is useful for comparing the phonetic similarity of two strings directly.

Syntax

DIFFERENCE(string1, string2)

Where string1 and string2 are the strings to compare.

Example

data soundex_comparison;
    input name1 $20. name2 $20.;
    diff = difference(name1, name2);
    datalines;
John Jon
Smith Smythe
;
run;

proc print data=soundex_comparison;
run;

In this example, the DIFFERENCE function compares the SOUNDEX values of "John" and "Jon", and "Smith" and "Smythe". Lower values indicate more similar phonetic representations.

The SPEDIS Function

The SPEDIS function measures the similarity between two strings based on the Soundex encoding and a variant of the Generalized Edit Distance. This function is useful for matching names with variations in spelling.

Syntax

SPEDIS(string1, string2)

Where string1 and string2 are the strings to compare.

Example

data spedisp_comparison;
    string1 = 'John';
    string2 = 'Jon';
    spedis_score = spedis(string1, string2);
    datalines;
;
run;

proc print data=spedisp_comparison;
run;

The SPEDIS function returns a score reflecting the similarity between "John" and "Jon". A lower score indicates higher similarity, similar to COMPGED, but with a different approach to similarity measurement.

Comparison of Functions

Here’s a quick comparison of these functions:

• SOUNDEX: Encodes a string into a phonetic code. Useful for phonetic matching, but limited to sounds and does not consider spelling variations.
• COMPGED: Uses the Generalized Edit Distance algorithm to measure string similarity. Suitable for fuzzy matching with spelling variations.
• DIFFERENCE: Compares the phonetic similarity of two strings based on their SOUNDEX values. Provides a direct measure of phonetic similarity.
• SPEDIS: Measures similarity using a combination of Soundex and Edit Distance. Useful for matching names with spelling variations and phonetic differences.

Conclusion

The SOUNDEX and COMPGED functions are valuable tools for text comparison in SAS. By understanding their characteristics and how they compare to other functions like DIFFERENCE and SPEDIS, you can choose the most appropriate method for your specific text matching needs. Each function offers unique advantages depending on the nature of the text data and the type of comparison required.