Today, we’re inundated with AI chatbot options. Every major vendor is in on the deal, from Amazon to Apple to Microsoft. Fortunately, there are a number of aggregator products that allows you to evaluate and compare different AI chatbot solutions. And these are not just a short-term evaluation tools; they can also be part of your daily-use toolbox.
Various products offer similar capabilities, and each has its own stand-out features. Before we look in-depth at one of them, GMTech, here’s a quick overview of the competitor landscape.
- Poe provides a customizable interface with its $20/month subscription. One of its key features is the ability to create customized chatbots for a more personalized and task-specific experience using multiple models. Poe’s website begins with a login prompt, so you must have an account to review the features.
- MrSeek is a new macOS-specific tool that integrates directly into your Mac’s toolbar, enabling access to multiple AI chatbots through your existing accounts. It relies on standard macOS keyboard shortcuts. As a standalone application, it’s lightweight and a more integrated solution than browser-based competitors. MrSeek’s target audience is Mac-based AI chatbot power users.
- InfernoAI offers several unique features to make daily use of AI chatbots more natural and useful. First, you have the standard option of selecting one or more of your favorite AI chatbot models. The list is as comprehensive as GMTech’s. The second interesting attribute is using characters to focus your queries.
- Multichat.ai offers a personable interface that provides a comfortable chatbot experience that is customizable for code generation, companionship, personal growth, and more. Organizations and individuals can customize Multichat for an easy-to-work interface with access to common AI models. You can choose from among three membership tiers, including Free, Premiere, and Pro.
A Look at GMTech
I’ve had a chance to look closely at GMTech, which is a good representative of what’s available in the AI chatbot evaluation tool market.
A subscription provides an interface to multiple AI chatbots seamlessly and simultaneously, enabling you to find the one that responds best to the type of query you’re sending. The user interface opens with two windows displayed. The first is Compare, enabling you to interact with multiple chatbots simultaneously. The second is Chat, a single-chatbot option for the times when you know which service you need or have limited screen space.
Switch between the two by using the Lab menu near your profile picture in the upper right corner.
I contrast the two choices below using a single basic query, “Explain how the Rocky Mountains were formed.”
GMTech Compare
GMTech’s AI Compare window opens with three default panes pointing to GPT-4o, Claude 3.5 Sonnet v2, and Gemini 1.5 Pro. An extensive menu pane along the left side allows you to select from various AI chatbot solutions listed alphabetically. Star your favorites after you begin using them. Twenty different AI chatbots from nine vendors were available at the time of writing.Add or remove the chatbots you’re interested in, and then use the message window at the bottom of the screen to start your conversation. Each chatbot pane contains its own spinning wheel, indicating it is performing its search and organizing the results. The GMTech interface uses the specific chatbot’s format to display results, so things can look a little different in each window. Be sure to use the hide feature to tuck the AI selection window out of the way for a little more real estate.
Remember, one use of this product is as an evaluation tool. GMTech displays how long each chatbot’s response took and the cost associated with it below the response. The measures are:
- Response time
- Response cost
- Conversation cost
You can also manage your own reaction to the responses by selecting a thumbs up (“I like it”) or a thumbs down (“Not good”). Finally, you have a button to copy the content.
Remember to select the chatbots you want to use before beginning your conversation. If you add a chatbot after messaging the others, it does not receive or respond to the query.
Use the temperature setting in the message window to dictate how predictable text is in the responses. Fine-tuning this setting can permit more creativity, which could be useful for certain query types. You can save or clear a conversation with chatbots individually by using the menu in the top right corner of each. This interface is optimized for use with desktops or tablets.
GMTech Chat
The Chat feature provides a similar look and feel to other online chat interfaces. The dialog goes back and forth in a familiar flow, encouraging you to ask the chatbot for additional details or more depth in specific areas. Like the Compare interface discussed above, each chatbot response includes the related response time and cost.
The message window allows you to select from among all available AI chatbots — a very comprehensive list. Note that it includes various tiered services, such as Anthropic’s Claude 3.5 Haiku, Claude 3.5 Sonnet v2, or Claude 3 Opus. You retain the options to save or clear the chat, define a temperature setting, and discard the query. The more focused Chat interface eliminates the distractions offered by multiple chatbots if you know exactly which model you prefer for a specific query. It is optimized for use with desktops, tablet devices, and phones.
GMTech Image Generation
GMTech also allows you to interact with a wide range of AI image generators. The interface resembles that of the Compare window. Describe an image in the messages bar at the bottom of the screen. Your images appear several moments later, including response time and cost data. My first query asked two AI image generators to “depict a starship from a dark science fiction future.” The results were both different and wonderful.
My next image generation attempt included more detail. It was, “Depict fantasy castle ruins in a jungle during daytime.”
Note the image cost and generation time information associated with each image.
Twelve image generators from seven vendors were available at the time of writing.
User Profile Settings
GMTech’s profile settings are basic, but then it’s not a platform that requires much personalization. Select your profile picture from the upper right corner, then choose the profile option. You can add your first and last name and upload a profile picture.
You can also access subscription information and help through the same menu.
Wrap Up
Overall, the interface is uncluttered and logical. Selecting AI chatbots in the Compare view is simple, even though GMTech offers access to many services. It’s a handy tool for evaluation teams trying to find the right AI chatbot for their company’s use. It’s also excellent for any chatbot power user juggling multiple subscriptions and trying to guess which chatbot will provide the best response for a given query.
The post Developer Tools To Evaluate and Compare AI Chatbots appeared first on The New Stack.
There are too many AI chatbots! Helpfully, tools like GMTech allow you to evaluate and compare different AI chatbot solutions.