Thursday, June 4, 2009

Why Can't My Grammar Checker Automatically Correct My Mistakes?

What Can We Expect, and Not Expect, from Grammar Programs?

Occasionally we receive a request for a program that will "read" a computer document and make all the grammar corrections. The requests usually sound something like this:

Does your software rewrite and organize awkward or ungrammatical sentences and paragraphs? Can it punctuate and present my document to make it clearer without changing the meaning?

On the surface that may sound like a reasonable question. Nowadays some of the spell checkers have instant correction modes--for example, if I type teh, it will automatically change it to the for me.

That may work with some common words we spell, but do not expect a grammar program to make editorial decisions.

Why not?

With my tongue in my cheek, I might answer, “First they must develop software that reads your mind!”

The concept is fairly simple, but the execution is anything but simple. We will not use the word impossible, but it is not going to be easy.

Grammar is Very Subtle

Grammatical relationships can be very subtle. In speech we often use things besides words to affect the grammar. The sentence "Your name is Mary," can be made into a question just by changing the tone at the end of the sentence: "Your name is Mary?" Other things like sarcasm and verbal irony are very difficult to program.

There are regional variations as well. In most areas, for example, people say, "I am waiting for you." However, in the southern United States, and among many African-Americans (whose roots are in the southern U.S.) people say, "I am waiting on you." Popular novelist John Grisham is from Mississippi and that is the way he writes. In other regions, to wait on means "to attend to, to serve," hence the words waiter and waitress.

There are also very subtle shades of meaning with compound words.

A parakeet is a blue bird.
A parakeet is not a bluebird.

When we read this or when we hear this, most native English speakers over the age of seven understand the differences. Would a grammar checker? It could, but it would not be easy. You would have to have about a dozen questions to ask for each such compound word, and there are hundreds, if not thousands of such compound words in English.

That is not all.

The Wrong Way to Write a Valentine

Here is a simple sentence punctuated two different ways. Both are correct, it depends on what you mean.

I think you are beautiful.
I think, you are beautiful.

I asked the girls in my grammar class, which would you want written on your Valentine card?

In the first, I is the subject and think is the verb. It is a straightforward declaration of the speaker's belief. Most women would consider that a compliment.

In the second, I think is a qualifying parenthetical expression showing some hesitation or doubt about the main clause, you are beautiful. One female student said to me, "I'd rather he lie than say that."

How is an automatic grammar checker going to be able to tell the difference?


Grammar Shows Intent

That is why grammar checkers may ask questions, give several choices, or give no choices. They do not know--no one does--what you intend to say. They will ask, “Which do you prefer?” Without getting too commercial, if you were not sure of the answer, you could quickly check in Grammar Slammer.

Here is another example. Again, both sentences could be punctuated correctly, but they mean two different things.

My aunt who lives in New York is coming to visit.
My aunt, who lives in New York, is coming to visit.

The first one specifies which aunt. The second one tells us something about the aunt, but the information about her being from New York is not critical to the meaning. That one illustrates a nonrestrictive modifier.

The first one can be helpful if you have several aunts and the person you are writing knows her as the one from New York. The second one suggests either that you only have one aunt, or that you have already established who it is you are writing about.


Family Trees and Automatic Checkers

For example, in my case, I only have one aunt. My father was an only child and my mother had just one sister. I would use the second one. My wife's mother, on the other hand, had seven siblings and her father had two. It took me years to get all the aunts and uncles straight; I still have not met a few of them. If she were trying to communicate which of the ten or so aunts she meant, she would use the first sentence. Do we ask grammar checkers to first search our family trees?

In fact, even that automatic spell checking feature is hardly foolproof. Yes, I appreciate the spell checker when it changes teh to the. But I am an English teacher, so sometimes I type out passages in older English which use the word thee. It gets annoying to have the automatic checker change thee to the! In that case, it is not being helpful!

There are many other difficulties as well, but this gives you an idea of some of the challenges of creating a computerized proofreader. Sure, spell checkers and grammar checkers make the job easier, but good publishers and printing companies still hire human proofreaders to complete the job.

Part 2—Grammar Affects Meaning

Grammar affects meaning. The reason is simple. Grammar is essential for communication. All languages have grammar rules, whether they are formal rules or informal patterns. If the basic rules are not followed, we are unable to communicate.

In a real sense words + grammar = communication.

When a correspondent wrote us back then if we knew of a grammar program that automatically corrected grammar errors, we replied, "Such a program will not exist until they develop software that reads minds."

Here is a simple example we encountered recently. A writer typed experiences shows for the subject and verb in a sentence. A grammar checker rightly identified this as a problem; the subject does not match the verb. Most grammar checkers can detect that error most of the time.

Why can’t someone simply add a feature so the grammar checker automatically corrects the mistake?

It is not that simple.

What does the writer intend? Does the writer want to make the subject singular or plural? Is the sentence supposed to say experiences show or experience shows? Both would be grammatically acceptable, but what is the writer's intent? Only the writer knows. A computer program doesn't. Even another person doesn't. So until we create mind-reading software, grammar checkers will continue to ask questions.

Part 3--What can grammar checkers really do for you?

At English Plus+ we probably get more questions asking about grammar checkers than anything else. Grammar checkers are useful tools. We at English Plus+ normally use one when we write anything we think might be important. At the request of users and our main commercial vendor, we have made a simple grammar checker ourselves. However, grammar checkers are limited. They are, in fact, more limited than spell checkers. The limitations have nothing to do with technology or software. The limitations are caused by the nature of grammar itself.

Grammar vs. Spelling
Grammar is much more complicated than spelling. Spelling is a matter of recognizing the order of letters in a word. A word either has the right letter pattern or it does not. Now there are some complications with spelling--some words have the same pronunciation but different spelling. Still, a good spell checker gives you several options to choose from. Some will even ask you about words that are commonly confused, like complement and compliment.

If you are really stuck, for example, and are not sure which "compliment" word is the one you need, you can check a dictionary or look it up in Grammar Slammer Deluxe which lists nearly 1200 commonly confused words. Checking spelling with a computer is really pretty simple. Grammar Slammer Deluxe makes it as convenient as possible.

Grammar is different. Grammar involves meaning and intent. We are all familiar with people who do not speak whatever our native language may be and are learning our language. They may put the right words together but what they say either makes no sense or is spoken in a way that no native would say it. That is because language is more than stringing words together; it is expressing thoughts by arranging words grammatically. And that grammar is unique in each language.
How Annoying Do You Want a Computer Program to Be?
Here is a simple example. In English, "John loves Mary" is not the same as "Mary loves John." The words are the same. The meaning of each word is the same. The grammar is different. Because the grammar is different, the meaning is different. If John loves Mary, he may hope that Mary loves John, but they are two different ideas.

A grammar checker that checks every little detail would be annoying. But that would insure that our meaning gets across. A programmer could design a grammar checker that asks, "Are you sure that you want 'John' to be the subject? Are you sure that John is doing the loving?" While probably no one would design a grammar checker to ask that question, there are many other questions a checker does ask.

Why Grammar Checkers Have to Let You Decide
Grammar checkers have to ask questions or let you decide. They cannot simply "correct a mistake." The reason is simple. Only you, the writer, know what you intended to say. Here is a simple example. Both sentences are grammatically correct. Even though they sound the same, they mean two very different things.

John loves Mary more than me.John loves Mary more than I.

This is the kind of thing that a grammar checker will often ask you about. Why? To make sure that you are saying what you intended to say. Yes, correct grammar is part of the issue, but the main part of the issue making your writing clear.

Only the writer knows whether John loves Mary more than He loves me or whether John loves Mary more than I love Mary.

An "Automatic Grammar Checker" Can't Read Your Mind
A grammar checker cannot simply "make corrections" the way that most spell checkers automatically change teh into the. If the checker does that, then it may be saying something different from what you mean to say. Normally, the word I follows the word than. It would be a simple matter to program a grammar checker so that it always changes than me into than I. That works fine when you write, “John is older than me." It should be “John is older than I.” The problem is that such a change would change the meaning of a sentence like "John likes Mary more than me." If the checker made the correction there, all of a sudden you are talking about how both John and the writer feel about Mary, not how they feel about each other!

Only you the writer know what you mean. A grammar checker can only guide you. You still have to decide. The main reason we originally wrote Grammar Slammer was to help people with those questions that the grammar checkers ask.
Perhaps, your grammar checker would ask you:

"Do you want 'John loves Mary more than me?' or do you want it so say 'John loves Mary more than I?"

Or the grammar checker might recommend than me, but you still have to decide whether to use it or not.

You might not be sure of which is better or which tells what you want to say. That is where Grammar Slammer comes in to help. You can quickly find this in the Grammar Slammer index under Than with Pronouns. There you would find what you need to know. There it would tell you:

Most of the times that we use a comparison with than or as, we leave words out. This is technically called an elliptical clause--a clause with an ellipsis. An ellipsis is words left out.

Look at it this way. There is a difference between the two following sentences. Both are grammatically correct; they just mean two different things.
He likes you more than me.
He likes you more than I.
Think of what words are left out:
He likes you more than I do. (I is the subject)
He likes you more than he likes me. (Me is the direct object)

Now you have something to go on. But can you see why the grammar checker has to ask a question and not simply "make a correction"? The grammar checker doesn't know your relationship to John and Mary! Only you do! Only you, the writer, can provide the meaning.

The grammar checker is a tool to help you convey the meaning. The grammar checker can help you make the meaning easier to understand. But ultimately, even with the best grammar checker, you still have to decide exactly what you mean and how best to say it.
Part 4—The Grammar Slammer Checkers

As mentioned before, we have developed a spelling checker and a grammar checker to go with Grammar Slammer Deluxe. Frankly, when we originally wrote Grammar Slammer and then Grammar Slammer Deluxe, we wrote them to assist users who needed a grammar or spelling reference or who wanted an easy tool to use when grammar or spelling checkers gave them a choice they were not sure about. However, over the years we have been asked about providing checkers. Now we have done this. With that in mind, we do want our users to look at things realistically.

First of all, we consider the Grammar Slammer and Spelling Slammer help files more important to users than our grammar and spelling checkers. These have the information which you need to learn to do things correctly. The most the checkers can do is a simple review.

We hope you have read our little examples above and understand that checkers have their limitations. Grammar checkers have even more limitations than spelling checkers do. They both are tools, but they both still depend on you, the user.

Most people understand how spelling checkers work. There is a dictionary file which contains correctly spelled words. The checker quickly scans a document and any word which it does not find in the dictionary, it automatically notes for the user. Some spelling checkers may have an automatic correction option.

When a problem word is noted, the user then has to decide whether to change the word to something that is in the dictionary, to ignore the word, or to add his or her spelling to the dictionary. This is where our original Grammar Slammer Deluxe can really help. It has a listing of many words and phrases which writers commonly confuse. Since it uses the Windows Help file format, most users find it easy to use and search.

Our spelling checker has a feature which takes you to the Spelling Index of Grammar Slammer Deluxe so you can quickly look up the word you have a question about. It makes Grammar Slammer Deluxe even more convenient.

Grammar checkers actually work in a way similar to spelling checkers. Instead of a dictionary, they have a pattern file with words or phrases that are commonly misused. When the grammar checker scans the document, it looks for such patterns. For example, the grammar pattern file might have the phrase kind of a because that normally indicates a grammar error. If a document had that phrase, the checker would flag it.

The depth of a grammar checker depends on how many such patterns there are in the pattern file, just as a spelling checker’s quality can be partly judged by how many words it has in its dictionary files. It also may explain why there are many spelling checkers available, but hardly any grammar checkers.

Many of us have had experiences with grammar checkers that may become annoying. Yes, I truly do know the difference between special and especial, for example, and a checker that constantly asks me which word I really mean begins to irritate me. No checker can avoid this entirely, but the checker that comes with Grammar Slammer does have thirty-one optional categories so it can skip over certain problems that you would catch on your own.

We have been told that competing with certain large software companies is the kiss of death. We have added the checkers to Grammar Slammer Deluxe because users have asked for them. Our checkers, especially the grammar checker, are relatively simple. We do not have a full-time team that continually adds to its pattern files or creates new dictionaries. No grammar checker will catch all errors. We will be happy to add any errors you find to our pattern files, but the grammar checker is a basic tool to help you. Its most appealing feature is that it also has a button that will take you to appropriate page or index page for many of grammar errors which it detects—even this feature is not as thorough as the same feature is for the spelling checker, but we have tried.

If you want a versatile tool that has checkers and references all together, then get Grammar Slammer Deluxe with Checkers. If you already have a spelling and grammar checker that you are satisfied with, but you need help in making decisions and corrections, then all you need is Grammar Slammer Deluxe. If your spelling is pretty good, then get basic Grammar Slammer—the original English grammar reference tool for Windows.

And may all your anguish be vanquished.
To order Grammar Slammer, Grammar Slammer Deluxe, or Grammar Slammer Deluxe with Checkers online, go to http://www.englishplus.com/gsorder.htm.


Copyright ©2002-2006 English Plus, All rights reserved.

1 comment: