Nvidia wants to strike a balance between accuracy and fairness in its AI models.

The company is seeking to patent a system for “fairness-based” neural network training using both real and synthetic data. This aims to make computer vision models more accurate and fair by creating more balanced datasets. 

Fairness in visual recognition models is a “recognized problem,” Nvidia said. While adding synthetic image data into these sets can help create a more balanced ratio of attributes, the “drawbacks are that the synthesized data typically has quality issues and may be biased.”

Nvidia’s system aims to fix this by finding the right ratio of synthetic and authentic data to train visual models to more accurately represent different attributes, particularly for gender, age, and skin color. The system first determines the real-versus-synthetic data ratio of the starting dataset, as well as the ratios of data from different attributes. 

Nvidia then uses “bilevel optimization,” or a math technique that solves two optimization problems that intersect. This technique balances accuracy and fairness in how much synthetic versus real data is used to represent different attributes. 

The system may also employ “group fairness measures” to avoid discriminating against “sensitive groups.” These fairness measures include equalized odds, ensuring an equal amount of false positives and false negatives across groups, and demographic parity, which ensures an equal amount of positive outcomes across groups. 

“Compared with conventional solutions, using real and synthetic training data improves accuracy while dynamically adjusting the sampling ratios improves fairness,” Nvidia said. 

Nvidia’s patent uses different words to solve a problem that many tech firms are trying to figure out: bias. Patent activity shows that the tech industry wants to take on AI’s bias issue in everything from lending to image models to training data itself. 

Nvidia dubbing its tech as a solution for “fairness” is a smart play to set itself apart in the eyes of patent reviewers – even if it is addressing the same issue as the other players, said Bob Rogers, PhD, co-founder of BeeKeeperAI and CEO of Oii.ai.

Focusing on facial recognition as its primary example is another smart angle. Facial recognition and analysis technology notoriously have issues with bias, with a higher error rate with darker-skinned individuals than lighter-skinned ones. And given that this technology is being used in security and policing contexts, “getting that bias out of there is critical,” said Rogers. 

“Protecting their own ability to develop tools, both in hardware and software that support facial recognition probably has a lot of value,” said Rogers. 

However, Nvidia relies on using synthetic data, which comes with its own caveats. While synthetic data gives developers access to a lot of data, it won’t have enough variation if it’s based on a small sample pool. For example, if you have a small data set of people wearing glasses, and all of the glasses are round, then the synthetic images will mirror that, and the image recognition model won’t know what to do when faced with square ones. 

“Because you’re still generating all those synthetic examples off of the basic examples that you have, you still can miss important attributes because you just simply don’t have enough data,” said Rogers. 

Nvidia’s tool is a great first step in fixing the bias problem, however, as it provides a litmus test to help developers figure out where exactly their dataset may be lacking, Rogers noted. “if your model changes a lot when you add synthetic examples from underrepresented sets, then you know that you’re focusing on things that are sort of group-centric.”

Hyundai wants to keep backseat drivers informed. 

The company is seeking to patent a system for “providing driving data” of an autonomous vehicle. Hyundai’s tech essentially relays information about a vehicle’s surroundings back to the occupant, keeping them in the loop whether they’re in the driver’s seat or not. 

Since autonomous vehicles drive with little human intervention, “an occupant may wonder whether the autonomous vehicle is accurately recognizing and determining the driving situation around the vehicle,” Hyundai said. This tech aims to put riders at ease. 

First, sensors positioned around the vehicle collect information about its surroundings, as these vehicles need to do anyway to operate properly. This could include awareness of pedestrians, road signs or conditions, vehicles, or other obstacles. 

The system then gathers information about the occupants inside the vehicle, including gaze direction, speech data, or movement. This is to get a sense of what the occupant may be seeing. For example, if a rider is looking out the window at pedestrians, or mentions something about the speed of the vehicle, this system would pick that up. 

Based on this, Hyundai’s tech then aggregates which information is pertinent for the occupant to know, such as local speed limits, surrounding landmarks, or potential hazards. The system would then project that information on the windshield where they can easily see it, “allowing the occupant to immediately check the main driving information and have trust in an autonomous system.”

Almost every Big Tech firm and automaker has thrown their hat in the self-driving car ring, whether it be for their own robotaxi or embedding autonomous technology into their own vehicles. Regulators, however, are catching up. 

While Tesla has long been the main subject of the National Highway Traffic Safety Administration’s ire, both Google-owned Waymo and Amazon-owned Zoox are being investigated by the agency. Waymo’s investigation follows 22 reports of its vehicles either crashing or violating traffic laws, whereas Zoox follows two crashes in which vehicles suddenly braked. 

These incidents don’t help the already shaky public perception of autonomous vehicles. One survey from AAA found 66% of US drivers expressed fear, and 25% uncertainty about fully self-driving cars. 

Distrust often extends beyond the person in the vehicle to other drivers, said Sean Stetson, VP of Engineering autonomous machines company Cyngn. “There’s still a level of disbelief that they have to overcome,” he noted. 

But tech like Hyundai’s aims to help strengthen that trust, giving drivers a sense of security that the vehicle understands the road in front of them, said Stetson. A lack of trust “is a huge blocker to autonomous vehicle adoption,” he said. 

However, another way to garner that trust is transparency, he said. For example, providing continued updates on development progress and reporting and rectifying incidents when they do occur may help assuage rider’s fears that the tech is unstable, he said. “That lack of transparency and openness and messaging definitely does erode trust.”

Google wants its robots to understand body language. 

The company wants to patent “detection and attention estimation” for human-robot interactions. Google’s tech wants to help robots with movement and decision-making by predicting and tracking the movements of the humans working beside them. 

As tech firms continue to tinker with robots in workplaces, operations, and day-to-day life, “it is becoming increasingly important for a robot to be aware of the intentions and expectations of these actors with respect to the robot,” Google said. 

To make these robots more aware, Google’s system captures images of the people in its periphery using cameras mounted on the device. This camera captures what Google calls “keypoints” of a person’s body, such as ankles, knees, hands, or elbows, that are moving to understand and predict their next move. It may also capture movements of the person’s face or eyes and collect audio to determine if the person is talking to it. 

Using a machine learning model, this system then predicts whether the person’s movements indicate that they’ll be interacting with the robot, and to what form that engagement may take. This would help the robot coordinate its own movements more intuitively and could prevent collisions or safety hazards. 

If certain keypoints on a person’s body aren’t visible to the camera, because they’re blocked by other objects the machine learning model may be trained to ignore them and extrapolate their positions, reducing the robot’s reliance on explicit image data. 

Tech giants seem very interested in making robots safer and smarter. Nvidia, Intel and Amazon have each sought several patents to make robots more aware and efficient in factory and logistics settings using AI. 

Outside of this patent, Google and its DeepMind AI division have both been hard at work, filing patents for tech to help robots better understand their surrounding environments and the humans within them. On the research side, the company is looking into things like a nearly indestructible robot hand and an AI model that translates language and vision into robotic actions. 

“While the applications of physical robotics are currently limited in scope, that is changing as we speak, and the further potential is expansive in areas such as manufacturing, logistics, transportation, and even inside consumer homes,” said Tejas Dessai, research analyst at Global X ETFs. 

These robots have massive potential in any space where human labor is involved, whether it be manufacturing, logistics, industrials, retail, or more. When properly trained, these devices can perform tasks precisely and quickly, allowing manufacturers to move to a fully automated, lights-out model. 

However, the costs of these systems remain a barrier, said Dessai. “Sophisticated robots are still expensive, and for real-world integration to play out, we need production volumes to go up significantly, which should drive unit prices down,” he noted. 

Plus, safety issues stand in the way of broader adoption. Because AI has the tendency to hallucinate and make mistakes, using it to work with heavy machinery in a fast-paced environment with human coworkers could create ample opportunity for accidents. However, patents like these that seek to read the body language of the people around them could alleviate those concerns and prevent collisions or safety risks.