ChatGPT's ability to provide conversational answers to any question at any time makes the chatbot a handy resource for your information needs. Despite the convenience, a new study finds that you may not want to use ChatGPT for software engineering prompts.  

Before the rise of AI chatbots, Stack Overflow was the go-to resource for programmers who needed advice for their projects, with a question-and-answer model similar to ChatGPT's. 

Also: How to block OpenAI's new AI-training web crawler from ingesting your data

However, with Stack Overflow, you have to wait for someone to answer your question while with ChatGPT, you don't. 

As a result, many software engineers and programmers have turned to ChatGPT with their questions. Since there was no data showing just how efficacious ChatGPT is in answering those types of prompts, a new Purdue University study investigated the dilemma. 

To find out just how efficient ChatGPT is in answering software engineering prompts, the researchers gave ChatGPT 517 Stack Overflow questions and examined the accuracy and quality of those answers. 

Also: How to use ChatGPT to write code

The results showed that out of the 512 questions, 259 (52%) of ChatGPT's answers were incorrect and only 248 (48%) were correct. Moreover, a whopping 77% of the answers were verbose. 

Despite the significant inaccuracy of the answers, the results did show that the answers were comprehensive 65% of the time and addressed all aspects of the question. 

To further analyze the quality of ChatGPT responses, the researchers asked 12 participants with different levels of programming expertise to provide their insights on the answers. 

Also: Stack Overflow uses AI to provide programmers new access to community knowledge

Although the participants preferred Stack Overflow's responses over ChatGPT's across various categories, as seen by the graph, the participants failed to correctly identify incorrect ChatGPT-generated answers 39.34% of the time.  

According to the study, the well-articulated responses ChatGPT outputs caused the users to overlook incorrect information in the answers. 

"Users overlook incorrect information in ChatGPT answers (39.34% of the time) due to the comprehensive, well-articulated, and humanoid insights in ChatGPT answers," the authors wrote. 

Also: How ChatGPT can rewrite and Improve your existing code

The generation of plausible-sounding answers that are incorrect is a significant issue across all chatbots because it enables the spread of misinformation. In addition to that risk, the low accuracy scores should be enough to make you reconsider using ChatGPT for these types of prompts. 

New York, Aug 13 (IANS) OpenAI’s ChatGPT answered about 52 per cent software engineering questions incorrectly, according to a study, raising questions about the popular language models accuracy.

Despite ChatGPT’s popularity, there hasn’t been a thorough investigation into the quality and usability of its responses to software engineering queries, said researchers from the Purdue University in the US.

To address this gap, the team undertook a comprehensive analysis of ChatGPT’s replies to 517 questions from Stack Overflow (SO).

“Our examination revealed that 52 per cent of ChatGPT’s answers contain inaccuracies and 77 per cent are verbose,” the researchers wrote in the paper, not peer-reviewed and published on a pre-print site.

Importantly, the team found that 54 per cent of the time the errors were made due to ChatGPT not understanding the concept of the questions.

Even when it could understand the question, it failed to show an understanding of how to solve the problem, contributing to a high number of conceptual errors, they said.

Further, the researchers observed ChatGPT’s limitation to reasoning.

“In many cases, we saw ChatGPT provide a solution, code, or formula without foresight or thinking about the outcome,” they said.

“Prompt engineering and human-in-the-loop fine-tuning can be helpful in probing ChatGPT to understand a problem to some extent, but they are still insufficient when it comes to injecting reasoning into LLM. Hence it is essential to understand the factors of conceptual errors as well as fix the errors originating from the limitation of reasoning,” they added.

Moreover, ChatGPT also suffers from other quality issues such as verbosity, inconsistency, etc. Results of the in-depth manual analysis pointed to a large number of conceptual and logical errors in ChatGPT answers. The linguistic analysis results showed that ChatGPT answers are very formal, and rarely portray negative sentiments.

Nevertheless, users still preferred ChatGPT’s responses 39.34 per cent of the time due to its comprehensiveness and articulate language style.

“These findings underscore the need for meticulous error correction in ChatGPT while also raising awareness among users about the potential risks associated with seemingly accurate answers,” the researchers said.

–IANS

rvt/prw

“There’s no me without Yohanes,” Yosief Kidane told Dateline, tearfully. “He’s my best friend in the world.”

Yosief’s brother, 22-year-old Yohanes Kidane, has been missing for almost a week. He was last seen on Monday, August 14, 2023, in San Jose, California.

“He’s always been right by my side,” Yosief said about the brothers’ closeness.

Yosief is the eldest of three siblings. “I’m a year and four months older than [Yohanes],” he said. “[Sara’s] three years younger than me.”

Their parents immigrated to the United States from Eritrea, but the siblings were born and raised in Rochester, New York.

Yosief told Dateline that he and his brother have always been extremely close. “People have always confused us. Our names, our appearance, our mannerisms,” he said. “We were fortunate to study together, grow up together, learn together.”

The two even went to the same college: Cornell University. “When he came in doing computer science, it kind of piqued my interest and made me a little competitive. So I started taking some classes and ended up really enjoying it,” he recalled. “Even though he’s a year younger than me, I had the privilege of being his homework partner in our algorithms class.”

Yosief told Dateline that Yohanes has always been smart and that’s something he said he’s always admired. “He was one of the best engineers out of his class at Cornell,” Yosief said. “Very smart, bright individual.”

In May 2023, Yohanes graduated from Cornell. Yosief said his brother soon moved to San Jose to start a job at Netflix as a software engineer. “He was super excited to start working at Netflix,” Yosief said. “He was always talking about how wise and capable his coworkers were.”

Yohanes had only worked at Netflix for two weeks before he disappeared.

Yosief, who lives in New York City, told Dateline he last spoke to his brother on Sunday, August 13. “He had work the next morning, but we talked for an hour,” he recalled. “He caught me up on a lot of stuff that was happening at work.”

The next day, Yohanes vanished.

Yosief told Dateline that on Monday August 14, their sister was checking Yohanes’s location and noticed it changed. “Somewhere around 8 p.m., she saw his phone location at the Golden Gate Bridge, which wasn’t anything out of the ordinary,” Yosief said. “Figured he could have been meeting a friend or, like, checking it out with maybe work people or something.” The Golden Gate Bridge is more than an hour away from Yohanes’s apartment.

By the next morning — Tuesday, August 15 — when Yohanes’s location still showed as being at the Golden Gate Bridge, Yosief said his sister started to panic. “She’d been calling, trying to see what he’s doing. He never picked up,” he recalled. “She calls me, wakes me up before I start work, and we start calling his phone, calling friends, trying to see where he could be.”

But no one had heard from Yohanes.

Yosief said that they later saw that the location of Yohanes’s phone started moving, so they called it. Yosief said a stranger picked up and said he had found the phone and Yohanes’s wallet, including his cash, card, and ID at the Golden Gate Bridge Welcome Center. “He said he would help us get them back to Yohanes and so we focused our efforts on actually finding out where he was,” he said.

Yosief told Dateline that they called the San Jose Police Department to conduct a wellness check at his brother’s apartment, but he wasn’t there. He said officers then went to Yohanes’s job and found out he never showed up to work that Tuesday. “This is where we start to really get scared,” Yosief said.

The San Jose Police Department Media Relations Unit told Dateline in an email, that Yohanes was reported missing to their department on the afternoon of Tuesday, August 15. The case was initially investigated by patrol officers and then turned over to detectives with the Missing Persons Unit. They were able to confirm that Yohanes used a ride share service to get from San Jose to San Francisco where he was last seen. 

According to Yosief, investigators learned that Yohanes got into an Uber, a black Toyota sedan, on Monday night around 7:15 p.m. outside of his apartment on N 4th Street in San Jose.

A spokesperson for Uber confirmed to NBC News that Yohanes had engaged their services and that his trip had “ended at the requested destination without any reported incidents.” They said the requested destination was the Golden Gate Bridge Welcome Center. The spokesperson said Uber has “been in touch with the driver and with law enforcement” to offer assistance.

Yosief said that investigators notified his family that same day that Yohanes’s backpack with his laptop and personal documents had been found. “We’re not exactly sure where, but we assume some proximity to the bridge,” he said.

It was then, according to Yosief, that his entire family flew from New York to the Bay area.  “My family started hanging posters around San Francisco, around medical centers, youth shelters, homeless shelters, anywhere you might have looked for aid or respite,” he said. “I walked around the bridge for hours.”

They found no sign of Yohanes.

The San Jose Police Department told Dateline they “are coordinating their investigation with the California Highway Patrol, the Golden Gate Bridge Highway and Transportation District, the United States Coast Guard, and [Uber] to obtain a conclusive answer as to Mr. Kidane’s whereabouts.” They added that there is no evidence to suggest foul play in the case at this time.

Yosief told Dateline that he won’t provide up looking for his brother. “We’re going to find him and we’re going to bring him home,” he said. “We’re not going to stop. We know our friends and family and the community is not going to stop.”

Yohanes is 5’8” and 150 lbs. He was last seen wearing gray sweatpants, a black hoodie, and black shoes.

Anyone with information about Yohanes’s disappearance is asked to call the San Jose Police Department at 408-277-0531.

Updated 8/22/23 to include comment from Uber and the San Jose Police Department.

That's ultimately the problem with all current generative AI—the things generated by these black box machines are more often than not, incorrect by any human reckoning. It's just that they look correct at a glance; they're "close enough" without thorough inspection. You can see it with images, with audio, and with text. While the tech is still improving, for now, you'd better stick to other humans when trying to learn a new skill.

Vagueness seems to be on the outs these days, particularly when it comes to prompt engineering and generative AI.

A lot of sage advice to newbies and novices that are using generative AI such as ChatGPT or Bard almost always includes the stern warning to avoid being vague in your prompts. Be specific, goes the bellowing mantra. Pick your words carefully and make sure to get to the meat of the matter. Do not be vague, you are forewarned with a nagging wagging finger pointed at you.

One supposes that this nearly overt hatred for vagueness comes partially from a long history of disdain for being vague. We aren’t just now on the alert for vagueness. It has been around for ages. For example, C.S. Lewis, the noteworthy British writer had said this about the dangers of being vague: “Always prefer the plain direct word to the long vague one.” The same qualm about vagueness extends into all arenas, such as this bashing of being vague as expressed by the legendary banker J.P. Morgan: “No problem can be solved until it is reduced to some simple form. The changing of a vague difficulty into a specific, concrete form is a very essential element in thinking.”

Okay, we get the message, loud and clear.

Vagueness is bad. And, by logical extension, being vague when using generative AI is clearly bad too. You can’t seem to beat that deductive reasoning. Any reasonably good prompting and any reasonably good prompt engineering strategy have to make sure to appropriately and dutifully alert you about the unsavoriness of vagueness when using generative AI.

Period, end of story.

Wait for a second, hold the presses!

Maybe there is a scintilla of redeeming value underlying the capacities of vagueness. Do we really need to toss the baby out with the bathwater? Perhaps vagueness can be harnessed and shaped to our advantage. Prompt engineering might be wiser to embrace the hidden gem of leveraging vagueness, doing so when suitable and when done with prompt-wise acumen.

In today’s column, I am continuing my ongoing special series on the latest advances in prompt engineering and will be covering the somewhat surprising revelation that vagueness definitely has an important place in your shrewd prompt engineering techniques and skills. I will be explaining what vagueness consists of and how it compares to the normative proclamation of always landing on the side of specificity. They are the yin and yang of prompting. Sometimes you ought to be using specificity, but not necessarily all of the time. Likewise, sometimes you ought to be using vagueness, but not necessarily all of the time.

Your aim with vagueness is to strive toward the Goldilocks goal, namely you don’t want to be too hot and not too cold. The proper balance of when and how to use vagueness in your prompts is the place you want to be. This is a mindful consideration. I say this because, with all the negative press about being vague, anyone that has gotten used to using generative AI might very well have formed a habit by now of eschewing vagueness like the plague. Well, if so, provide vagueness a second chance. You’ll be glad you did.

Here's how we will proceed herein.

First, I’ll discuss the nature of vagueness and what we as humans don’t like about it overall. The dislike of vagueness will next be tempered by showcasing the strength and importance of being vague. I will then dive into the role of vagueness within prompts and generative AI all told. If all goes well, the wheels will get spinning in your mind as to leveraging vagueness on a daily basis when you are using generative AI.

In defense of vagueness, there are some famous quotes that provide a happy-face perspective on being vague. John Tukey, the famous mathematician, said this uplighting remark about vagueness: “Far better an approximate answer to the right question, which is often vague, than the exact answer to the wrong question, which can always be made precise”.

Keep in mind too that one of the most powerful elements of being vague is that it can be a boon to creativity, as well stated by the renowned painter Pablo Picasso: “You have an idea of what you are going to do, but it should be a vague idea.” Let’s not rigidly bash vagueness and instead see what it is in a fuller picture for all that it portends, opening our eyes wide to see both the bad and the good at hand.

Before I dive into the crux of leveraging vagueness as an indispensable tool of prompt engineering, let’s make sure we are all on the same page when it comes to the keystones of prompt engineering and generative AI.

Prompt Engineering Is A Cornerstone For Generative AI

As a quick backgrounder, prompt engineering or also referred to as prompt design is a rapidly evolving realm and is vital to effectively and efficiently using generative AI or the use of large language models (LLMs). Anyone using generative AI such as the widely and wildly popular ChatGPT by AI maker OpenAI, or akin AI such as GPT-4 (OpenAI), Bard (Google), Claude 2 (Anthropic), etc. ought to be paying close attention to the latest innovations for crafting viable and pragmatic prompts.

For those of you interested in prompt engineering or prompt design, I’ve been doing an ongoing series of insightful looks at the latest in this expanding and evolving realm, including this coverage:

• (1) Practical use of imperfect prompts toward devising superb prompts (see the link here).
• (2) Use of persistent context or custom instructions for prompt priming (see the link here).
• (3) Leveraging multi-personas in generative AI via shrewd prompting (see the link here).
• (4) Advent of using prompts to invoke chain-of-thought reasoning (see the link here).
• (5) Use of prompt engineering for domain savviness via in-model learning and vector databases (see the link here).
• (6) Augmenting the use of chain-of-thought by leveraging factored decomposition (see the link here).
• (7) Making use of the newly emerging skeleton-of-thought approach for prompt engineering (see the link here).
• (8) Determining when to best use the show-me versus tell-me prompting strategy (see the link here).
• (9) Gradual emergence of the mega-personas approach that entails scaling up the multi-personas to new heights (see the link here).
• (10) Discovering the hidden role of certainty and uncertainty within generative AI and using advanced prompt engineering techniques accordingly (see the link here).
• (11) Additional coverage including the use of macros and the astute use of end-goal planning when using generative AI (see the link here).

Anyone stridently interested in prompt engineering and improving their results when using generative AI ought to be familiar with those notable techniques.

Moving on, here’s a bold statement that pretty much has become a veritable golden rule these days:

• The use of generative AI can altogether succeed or fail based on the prompt that you enter.

If you provide a prompt that is poorly composed, the odds are that the generative AI will wander all over the map and you won’t get anything demonstrative related to your inquiry. Being demonstrably specific can be advantageous, but even that can confound or otherwise fail to get you the results you are seeking. A wide variety of cheat sheets and training courses for suitable ways to compose and utilize prompts has been rapidly entering the marketplace to try and help people leverage generative AI soundly. In addition, add-ons to generative AI have been devised to aid you when trying to come up with prudent prompts, see my coverage at the link here.

AI Ethics and AI Law also stridently enter into the prompt engineering domain. For example, whatever prompt you opt to compose can directly or inadvertently elicit or foster the potential of generative AI to produce essays and interactions that imbue untoward biases, errors, falsehoods, glitches, and even so-called AI hallucinations (I do not favor the catchphrase of AI hallucinations, though it has admittedly tremendous stickiness in the media; here’s my take on AI hallucinations at the link here).

There is also a marked chance that we will ultimately see lawmakers come to the fore on these matters, possibly devising and putting in place new laws or regulations to try and scope and curtail misuses of generative AI. Regarding prompt engineering, there are likely going to be heated debates over putting boundaries around the kinds of prompts you can use. This might include requiring AI makers to filter and prevent certain presumed inappropriate or unsuitable prompts, a cringe-worthy issue for some that borders on free speech considerations. For my ongoing coverage of these types of AI Ethics and AI Law issues, see the link here and the link here, just to name a few.

With the above as an overarching perspective, we are ready to jump into today’s discussion.

Foundations Of Vagueness And The Value To Be Derived

We will ease our way into the role of vagueness, doing so by first examining how humans convey and perceive the nature of being vague. After we cover that aspect, we can then consider how humans interacting with generative AI are likely to act and react related to how the AI does or does not express vagueness.

It is useful to first explore how humans express and react to vagueness when interacting with fellow humans. Here’s why. When we use generative AI, we tend to carry over our preexisting assumptions and habits about vagueness that have been dutifully learned or naturally acquired throughout our lives on a human-to-human interaction basis.

I cover the matter in this way with a bit of erstwhile caution because I don’t want anyone to be led down the path of anthropomorphizing AI. In current times, AI is not sentient and should not be equated to the sentience of humans. I will do my best to make that same alert when we get into certain aspects of the generative AI details that might seem overly sentient-like.

Thanks for keeping a level head on these weighty matters.

Let’s start with the meaning of vagueness.

In a research article entitled “Vague Language and Its Social Role” by Mubarak Alkhatnai, published in the Theory and Practice in Language Studies, February 2017, these keystones about vagueness are laid out:

• “Vague language denotes phrases and words that are neither exact nor precise.”
• “People often use these phrases in cases where they are not sure about something, to save time during a conversation, and to speak informally but in a manner that is friendly.”
• “Vague language is pervasive in everyday talk serving interpersonal and pragmatic functions in discourse.”
• “The use of vague language is a common phenomenon in any given society or cultural setting.”

The gist here is that vagueness in our everyday lives is always present. It is all around us. We are immersed in a world of great vagueness. The trouble arises when you are expecting specificity but get mismatched with vagueness.

Suppose that someone tells you that a person is tall. What do you think the actual height of the person consists of?

It is quite hard to discern from the vague wording entailing tallness. If children tell you that someone is tall, you likely would suspect that the tallness is not much of any notable magnitude since the kids are likely comparing their height to the height of the person being described. On the other hand, if you knew that the person was a professional basketball player, you would undoubtedly reasonably guess that they are indeed relatively tall (online reported stats suggest that such pros are around 6 ½ feet tall, surpassing the average non-player by about 8 inches).

This showcases that some words are inherently vague. We often find ourselves having to cope with vague words and discern what they mean or how to best interpret them. Another facet is the context of the vagueness makes a big difference too.

Imagine that I tell you that someone is 6.2 feet tall. By all appearances, the fractional portion suggests that this is a very specific accounting for the height of someone. We would assume that 6.2 is rather precise.

But that might not really be the case. Pretend that we have a room full of people and they are all perchance 6.2 feet in height (perhaps it is an annual convention of people that are that specific height). They decide to have a contest to see which of them is the tallest. One person yells out that they are 6.2 feet tall. So does the next person, and so on. All of them are that height.

In a sense, 6.2 feet is vague in this context. It is not fully specific. We might measure each person and reach a greater specificity. One person is 6.24 feet tall. Another one is 6.28 in height. This gets us away from the vagueness of 6.2 and toward a more specific accounting.

I bring this all up due to the important consideration that vagueness is often in the eye of the beholder. A given word or phrase is not axiomatically specific and ergo non-vague. Things depend upon what the context consists of and a variety of other mitigating factors that come to play.

One of the most famous examples of this nebulousness about vagueness-versus-specificity comes in the classic paradox-of-the-heap or also known as the sorites paradox (side note, the word “sorites” comes from the original Greek word for a heap).

Here’s how this paradox goes.

You are told that a heap of sand is defined as 1,000,000 grains of sand. This seems plain and simple to understand. Go ahead and assume that the definition is valid, and we all agree that a heap is one million grains of sand.

Another premise or precept that you are told is that taking one grain from a heap of sand is considered relatively inconsequential and therefore you still have a heap of sand. This seems sensible. If we had 1,000,000 grains of sand and took out one grain, the resulting 999,999 grains of sand are certainly going to appear to be like the same heap we had a moment ago. Unless you have some miraculous and uncanny ability to discern the omission of one grain of sand, the heap sure looks still like a heap.

Now the mental trap is set.

Another grain of sand is removed. Does the heap remain as a heap? I suppose you would still believe that a mound of 999,998 grains of sand is pretty much still a heap. You might shrug your shoulders and say that yes, we still have a heap.

Take away another grain. And another grain. All along, you are still seemingly going to say that this is still a heap of sand. Aha, this removal of a grain at a time keeps taking its toll. Eventually, we get down to one grain of sand left.

Would you say that one grain of sand is a heap of sand?

Most people would say that this is not a heap of sand. It seems rather ridiculous to suggest otherwise. But the logic of our approach appears to box us in and we must acknowledge that the one grain of sand is a heap. We step by step kept saying that we still had a heap.

The paradox has us over a barrel. Part of the reason that we are in a pickle is that we might have originally presumed that the definition of a heap was particularly specific. We were told a heap has a million grains of sand. We didn’t start by saying that a heap is a bunch of sand or a collection of sand. The specifics seemed to shine brightly by telling us explicitly that it was a million grains of sand.

The additional rule that said we could take away a single grain and yet still have a heap also seemed quite specific. Unfortunately, this rule has landed us in trouble. We have no specificity as to how many single grains of sand being removed will ultimately toss us out of the heap category. If the rule had said that once you remove say 10,000 grains of sand you no longer have a heap, we would have something more specific to work with.

I trust that you enjoyed that enlightening paradox.

What if we required everyone to be exactingly specific in everything they utter?

Would this fix our daily problem of dealing with vagueness?

Not quite.

You could find yourself exhaustingly having to prattle on and on to try and overcome the vagueness involved. The amount of specificity could almost be endlessly pursued. There is also the likelihood that the specificity isn’t especially helpful to the situation.

Going back to the notion of someone being tall, their tallness might not be an important element and thus trying to nail down the specifics might be a waste of time and effort. For example, if you are told that a person is tall and running away from an angry dog, you might not care about the tallness and be more interested in the running and the matter of the growling dog that is in pursuit. If the tightening of the phrasing about tallness is needed, perhaps this can be dealt with later on.

You see, we can reasonably assert and compellingly proclaim that vagueness does in fact have redeeming qualities.

Our communications with other people can be more efficiently stated by avoiding the grinding delineation of specifics. Using vagueness is bound to (usually) be a faster way to convey something. There is also the possibility that the specificity is unknown and thus the vagueness is used as a placeholder. Maybe we don’t know exactly how tall the person is, but we generally perceive them as tall. This is useful as a placeholder in lieu of knowing the precise height.

There are occasions where you purposefully use vagueness and knowingly do so. I’d bet that you’ve done this many times. You might have been telling a story about an encounter with a tall person but the tallness wasn’t a vital factor. The assertion they were tall is sufficient and no further specificity is required.

One would suppose that there are also occasions where you inadvertently use vagueness and didn’t do so with an intention in mind. You are telling the story about the tall person, and your friend listening to the story stops you. What do you mean by being tall? You realize that you probably should have been more specific. It was not something top of mind.

The same rules apply to being specific. There are occasions where you purposefully use specificity and knowingly do so. Your telling of the story about the tall person might be that you say they are 6 feet 7 inches tall. You want to convey the magnitude or degree of tallness. This impresses your friend that is listening to the story.

On other occasions, you might use specificity and do so without an intention in mind. During the telling of the story about tallness, you mention that the person was 6 feet 7 inches tall. Your friend stops you. How do you know they are exactly that height? This raises a host of questions. You realize that this is a distraction from what you were trying to say. Oops, you probably should have been vaguer and said they were simply tall.

It is safe to say that vagueness has a tradeoff and requires a kind of mental calculation as to the benefits and costs associated with being vague (likewise the same for being specific). There is an ROI (return on investment) underlying the use of vagueness.

We need to be aware of vagueness and specificity. We ought to leverage either one to the circumstances at hand. To say that either one is outrightly bad or wrong to use is a false proposition. They are both tools. Use the right tool in the right way at the right time and place.

With all of that now under our belt, we are ready to see how this applies to prompts, prompt engineering, and the use of generative AI.

Prompt Engineering And The Use Of Vagueness

When you enter a prompt into generative AI, you are oftentimes wanting to get a somewhat focused response generated by the AI. One issue with generative AI is that it is like a box of chocolates, whereby you never know exactly what the generative AI will respond with. You can ask quite a specific question and get a vague answer. You can even get an answer that seems far afield from your question, especially if the generative AI has a so-called AI hallucination (as mentioned earlier and as covered in my column at the link here).

The rule of thumb is that you should be as specific as you can in your prompt. The hope is that the more specificity there is will guide or spur the generative AI to be specific in return. This is a good practice. If your prompt is meandering or confusing, the generative AI is going to have a tough time pattern-matching what you want to find out about.

You can even urge the generative AI to explicitly be specific in its response. If you have a prompt that asks generative AI to identify how to wash a car, you can include in your prompt an indication that you want a listing of five steps that are undertaken when washing a car. The odds are pretty high that the generative AI will respond with five steps.

The problem though is that you might be cornering or constraining the generative AI due to your use of specificity in your prompt.

Suppose that the generative AI would have come up with seven steps regarding how to wash a car. But you told the generative AI to specifically come up with five steps. Your specificity might have shot your own foot. Perhaps there really are seven steps and you’ve now gotten the generative AI to curtail or cut off the seven by reducing the steps to five.

The five steps might be fine and upon comparing to the seven steps you don’t see anything significant that was left out. On the other hand, it could be that the generative AI dropped out two crucial steps and landed on five that aren’t as crucial. This is all going to be context-dependent.

If you had been vague and merely said tell me the steps involved in washing a car, you are generally allowing the generative AI to choose the number of steps. Doing so could be very handy for you. Upon the steps being presented to you, you could subsequently tell the generative AI to make the steps into just five (assume that seven were presented).

Which is better, should you be vague in your prompt or be specific?

There isn’t any singular right answer since the context and what you are trying to do will determine the choice of being vague versus being specific.

For those that are just starting with generative AI, the guidance of being specific is a good one. These novices or newbies will likely be more fulfilled by the responses of the generative AI as a result of being specific in their prompts. This is reassuring.

Akin to riding a bike, instructions about how to ride a bicycle often simplify the world when you first get underway. Once you’ve gotten comfortable riding a bike, you begin to find shortcuts or other ways to ride. More advanced riders also might share their insights about how to further boost their bike riding prowess.

The same goes for vagueness and specificity.

Your best bet when starting with generative AI would be to lean into specificity. Once you have your sea legs under you, you can branch out and leverage vagueness. Anyone of advanced prowess in prompt engineering should purposefully be choosing either vagueness or specificity as befits the situation at hand.

Here are four useful ways to think about this:

• (1) Vagueness at the get-go. You word your prompt for vagueness and that’s all you have in mind.
• (2) Vagueness followed subsequently by specificity. You word your prompt for vagueness and have in mind that depending upon the generated response you might likely enter a subsequent prompt entailing specificity.
• (3) Specificity at the get-go. You word your prompt for specificity and that’s all you have in mind.
• (4) Specificity followed subsequently by vagueness. You word your prompt for specificity and have in mind that depending upon the generated response you might likely enter a subsequent prompt entailing vagueness.

The first three of the above recommended practices are perhaps self-evident and I have explained them in my example about washing a car. The fourth one might be a bit puzzling to you. You might be wondering when you would ever start with specificity and then drive the generative AI toward being vague.

We can use the car washing example again. You tell generative AI to provide you five steps. It does so. You might at that juncture be satisfied and move on. There might though be a small concern in your mind that maybe the five steps are insufficient. You were the one that constrained the generative AI. You don’t know whether this was a suitable number of steps or not.

With that suspicion in hand, you opt to do a follow-up prompt and tell the generative AI to not be constrained with just the stated five steps. This new prompt pushes the generative AI to more freely provide an answer, a vaguer answer.

Upon inspecting the vaguer answer, perhaps this reveals that the generative AI came up with seven steps and reduced things down to five. There is another possibility that the generative AI does an overreach and comes up with seven steps even though only five are sufficient. Sometimes generative AI will respond to a prompt by seemingly attempting to fulfill a request despite there not being a need per se to do so. Rarely will you get generative AI retorting that there are five and only five steps. If you seem to hint or suggest that some other number is viable, this might be contrived or concocted for you by the pattern-matching of the generative AI.

Thinking Bigger About Vagueness When Prompting

I’ve been discussing the use of vagueness in your prompting strategy. Turns out that is one side of the coin. The other side is whether the response by generative AI is vague or not.

We have these four notable conditions arising from two overarching principles:

• (1) Vagueness in your prompt
• a. Intentional vagueness in your prompt. You are intentionally vague in your prompt.
• b. Unintentional vagueness in your prompt. You are vague in your prompt but did not intend to do so.
• (2) Vagueness in the AI response
• a. You request vagueness in the AI response. You tell the generative AI to be vague in the generated response and it will probably do so.
• b. You omit what you want in terms of vagueness. You do not tell the generative AI to be vague in the generated response and the resulting response might be vague or might not be (roll of the dice by the computational whims of the generative AI).

Let’s briefly walk through those.

The first point of the four listed points, namely #1a, further emphasizes my above discussion about the value of intentionally using vagueness in your prompts. Vagueness is a tool. Use the tool wisely.

The second point, labeled as #1b, refers to unintentionally using vagueness in prompts. I would guess that many users of generative AI are blissfully unaware of the idea that being specific or vague in their prompt might make a difference in how the AI will respond. Thus, they by and large are at times vague even though they aren’t intentionally doing so. It just happens. To them, I wish them luck but also urge that they cognitively consider the wording of their prompts and leverage vagueness and specificity when suitable to do so.

The third point, #2a, is something I have alluded to earlier, but we can now cover it directly at this juncture. You can consider stating in your prompt whether you want generative AI to respond with a specific answer or a vague answer. I suppose asking or telling AI to be vague seems counterintuitive to you. The thing is, as noted about car washing, you might lead the AI down the path of being specific when the better answer might be a vaguer one. It is usually best to distinctively ask for what you want.

The fourth point, #2b, is that if you don’t say anything either way in your prompt about how you want the generative AI to respond (regarding vagueness or specificity), the AI might go either way. Sometimes it might be specific, sometimes it might be vague. A roll of the dice is underway.

My recommendation is that you seek to:

• Abide by #1a (intentionally using vagueness in your prompt, when warranted),
• Seek to avoid #1b (by unintentionally using vagueness in your prompt you are essentially falling asleep at the wheel)
• Make use of #2a (intentionally tell the AI to be vague when that’s what you want)
• Cautiously do #2b (failing to say whether you want vagueness or not is probably okay most of the time, since this is likely a pinpoint consideration more so than an overarching one).

Try to wrap those recommendations into your prompt engineering mindset. The goal is to comfortably use those pieces of salient advice. They should become second nature.

As an overall indication of why you might want vagueness in your prompt, I offer these possibilities:

• Use vagueness when you want to go on a fishing expedition for what you might uncover.
• Use vagueness because you aren’t sure of what to ask or how to ask about your problem at hand.
• Use vagueness to overcome the narrowness that might arise from using specificity.
• Use vagueness because it is faster or easier than being more specific.
• Etc.

And here’s an overall indication of why you might want vagueness in the generated response from generative AI:

• Ask for vagueness in the response to get an overall less-constrained response (maybe you’ll get this, maybe not).
• Ask for vagueness in the response so you can compare it to a specific indication that you already have in hand.
• Ask for vagueness in the response to get the generative AI nudged out of a potential pattern-matching pothole that it has landed in.
• Ask for vagueness in the response to get a chain-of-thought or skeleton-of-thought prodding that you then follow up with specificity (see my discussion at the link here and the link here).

Those tips and remarks might inspire you to leverage vagueness from time to time.

Conclusion

I’ve got a final twist for you on this.

The characterization of vagueness-versus-specificity tends to suggest that they are exclusively used on a one-at-a-time basis. That is a misnomer.

You can use them both at the same time.

When you compose a prompt, there are undoubtedly some aspects that you prefer to have vaguely stated and other aspects that it makes more sense to distinctly specify. The same goes for the response that you want from the generative AI. You might readily want the generated response to contain some aspects that are vague and other aspects that are very specific.

I am pleased to report that you can at times have your cake and eat it too.

For those portions of your prompt that you believe are best conveyed vaguely, do so. Other portions of your prompt might contain a great deal of specificity. Mix and match as warranted.

For the generated response by AI, you might tell the AI which aspects should be vaguely stated and which should be stated with specificity. You can ask for both too. This could consist of telling the AI to identify how to do a car wash in five steps, and simultaneously having it in the same response list any number of steps that might be suitable to do so. I ought to forewarn you that this simultaneous duality of vagueness and specificity can impact what the generative AI produces in the sense that asking for five could steer the AI in a direction that a vaguer indication would not have. Please keep that caveat in mind.

Let’s end this discussion with an insight into where vagueness can take you. Bertrand Russell, eminent mathematician and grand philosopher said this about being vague: “Everything is vague to a degree you do not realize till you have tried to make it precise.”

That’s the beauty of leveraging vagueness when using generative AI. Your effort to consider the vagueness versus specificity facets is likely to get you thinking more deeply and clearly about what you are wanting to use generative AI for. You will need to confront the at times paradoxical ramifications of what is vague and what is specific.

While you ponder those hefty thoughts and begin to explicitly carry on with these prompting techniques associated with vagueness, I’ll be counting grains of sand. I’ll let you know once I run out of a heap.

starnewsonline.com cannot provide a good user experience to your browser. To use this site and continue to benefit from our journalism and site features, please upgrade to the latest version of Chrome, Edge, Firefox or Safari.

• Chrome: https://www.google.com/chrome/
• Edge: https://www.microsoft.com/edge/
• Firefox: https://www.firefox.com/
• Safari: Update to latest version through the App Store

This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

ACM TechNews

By ZDNET
August 16, 2023
Comments

ChatGPT answered 52% of 517 Stack Overflow questions incorrectly, and 77% of its answers were unnecessarily wordy, according to in a study by Purdue University researchers.

The study found that ChatGPT gave comprehensive answers to software engineering questions, addressing all aspects of the question, 65% of the time.

Researchers also asked 12 individuals with different programming skill levels to assess the ChatGPT-generated answers. "Users overlook incorrect information in ChatGPT answers [39.34% of the time] due to the comprehensive, well-articulated, and humanoid insights in ChatGPT answers,” the researchers say.

From ZDNET
View Full Article

Abstracts Copyright © 2023 SmithBucklin, Washington, D.C., USA

---

No entries found