Creating Interactive 3D Anatomy Models for Fitness Apps with React Three Fiber

In the competitive world of fitness technology, user engagement and education are paramount. While countless apps can track reps and sets, few offer a truly intuitive way to understand the why behind an exercise. What if you could visually show your users that a bicep curl targets the brachialis and brachioradialis, not just the biceps brachii?

This tutorial will guide you through building an innovative feature: an interactive 3D human anatomy model that highlights the specific muscles activated during a selected exercise. We'll leverage the power of React Three Fiber, a declarative and component-based renderer for Three.js, to bring a 3D model to life directly in your web application.

By the end of this guide, you'll have a reusable React component that not only enhances your app's educational value but also provides a "wow" factor that sets you apart.

Prerequisites:

• Basic understanding of React and JavaScript.
• Node.js and npm (or yarn) installed.
• A 3D anatomy model in .gltf or .glb format. We'll discuss where to find one.

Why this matters to developers:

Integrating 3D elements into web applications is a rapidly growing trend. Mastering libraries like React Three Fiber opens up new possibilities for creating immersive experiences in e-commerce, data visualization, and, as we'll see, health and fitness. This project is a practical and impressive addition to any portfolio.

Traditional fitness apps often use static images or videos to demonstrate exercises. While helpful, they lack interactivity and can't always convey the nuances of muscle engagement. Our goal is to solve this by creating a dynamic, 3D visualization where users can:

1. See a full 3D human model.
2. Select an exercise from a list (e.g., "Bicep Curl," "Squat").
3. Instantly see the primary and secondary muscles for that exercise glow on the 3D model.

This approach transforms a passive learning experience into an interactive one, helping users connect with their bodies and improve their workout efficacy.

Before we start coding, let's set up our development environment and find a suitable 3D model.

1. Project Setup

Start by creating a new React application:

npx create-react-app r3f-fitness-anatomy
cd r3f-fitness-anatomy

Next, install React Three Fiber and its helper library, @react-three/drei, which provides useful abstractions and components.

npm install three @react-three/fiber @react-three/drei

2. Finding a 3D Anatomy Model

A crucial part of this project is the 3D model itself. For the highlighting effect to work, you need a model where individual muscles are separate meshes. Here are some resources for finding free or open-source models:

• Sketchfab: A great platform with many downloadable 3D models. Look for models with a permissive license.
• OpenAnatomy: A project dedicated to creating and sharing open-source anatomy models.
• Z-Anatomy: Offers downloadable .blend files that can be exported to .gltf.

For this tutorial, we'll assume you've found a .glb or .gltf model and placed it in your public/ directory. For example: public/models/anatomical_model.glb. A key requirement is that the model's meshes have descriptive names (e.g., "bicep_brachii_left", "pectoralis_major_right"). You can inspect and rename meshes using a 3D modeling tool like Blender.

First, let's create the 3D space where our model will live. React Three Fiber provides a <Canvas> component that sets up the scene, camera, and renderer.

What we're doing

We'll replace the default content of App.js with a basic R3F scene, including lighting and camera controls.

Implementation

// src/App.js
import React from 'react';
import { Canvas } from '@react-three/fiber';
import { OrbitControls } from '@react-three/drei';
import './App.css';

function App() {
  return (
    <div className="App">
      <Canvas camera={{ position: [0, 0, 3], fov: 60 }}>
        {/* Lights */}
        <ambientLight intensity={0.5} />
        <directionalLight position={[2, 5, 2]} intensity={1} />
        
        {/* Controls */}
        <OrbitControls />
        
        {/* Our 3D Model will go here */}
      </Canvas>
    </div>
  );
}

export default App;

And add some basic styling to make the canvas full-screen:

/* src/App.css */
.App {
  width: 100vw;
  height: 100vh;
  background-color: #2c3e50;
}

How it works

• <Canvas>: This component from @react-three/fiber is the entry point for all 3D content. We've configured the initial camera position and field of view (fov).
• <ambientLight> & <directionalLight>: Lighting is essential in a 3D scene. Without it, your model will appear black. An ambient light provides a soft, global illumination, while a directional light simulates a distant light source like the sun.
• <OrbitControls>: A helper from @react-three/drei that allows users to rotate, pan, and zoom the camera with their mouse.

Run npm start, and you should see a dark background. You can click and drag to interact with the scene, but there's nothing in it... yet!

Now for the exciting part: bringing our 3D model into the scene.

What we're doing

We'll create a new component for our anatomy model, load the .glb file using a hook from @react-three/drei, and render it.

Implementation

First, let's convert our .glb model into a reusable React component. The creators of React Three Fiber have made a fantastic online tool for this: gltfjsx.

1. Go to https://gltf.pmnd.rs/.
2. Drag and drop your .glb file onto the page.
3. Copy the generated JSX code.

Now, create a new file src/AnatomyModel.js and paste the code. It will look something like this (simplified for clarity):

// src/AnatomyModel.js
import React from 'react';
import { useGLTF } from '@react-three/drei';

export function AnatomyModel(props) {
  const { nodes, materials } = useGLTF('/models/anatomical_model.glb');
  
  return (
    <group {...props} dispose={null}>
      <mesh
        name="bicep_brachii_left"
        geometry={nodes.bicep_brachii_left.geometry}
        material={materials.muscle_material}
      />
      <mesh
        name="pectoralis_major_right"
        geometry={nodes.pectoralis_major_right.geometry}
        material={materials.muscle_material}
      />
      {/* ... all other muscles */}
    </group>
  );
}

useGLTF.preload('/models/anatomical_model.glb');

Now, let's render it in App.js:

// src/App.js
import React, { Suspense } from 'react';
import { Canvas } from '@react-three/fiber';
import { OrbitControls } from '@react-three/drei';
import { AnatomyModel } from './AnatomyModel'; // Import the model
import './App.css';

function App() {
  return (
    <div className="App">
      <Canvas camera={{ position: [0, 0, 3], fov: 60 }}>
        <ambientLight intensity={0.5} />
        <directionalLight position={[2, 5, 2]} intensity={1} />
        
        <Suspense fallback={null}>
          <AnatomyModel />
        </Suspense>
        
        <OrbitControls />
      </Canvas>
    </div>
  );
}

export default App;

How it works

• gltfjsx: This command-line tool (and web interface) analyzes your GLTF file and generates a JSX component that represents the model's structure. This is incredibly powerful as it turns your model into a declarative React component.
• useGLTF: This hook from @react-three/drei loads the model and gives you access to its nodes (the geometry of each part) and materials.
• <Suspense>: Model loading is asynchronous. React's <Suspense> component allows us to provide a fallback (like a loader) while the model is being fetched. For now, we're just showing null.

You should now see your 3D anatomy model rendered in the browser!

This is where we add the core functionality. We want to highlight specific muscles based on a selected exercise.

What we're doing

We will create a simple data structure to map exercises to muscle names. Then, we'll modify our AnatomyModel component to accept a list of muscles to highlight and change their material properties accordingly.

Implementation

First, let's define our exercise data. Create a new file src/exerciseData.js:

// src/exerciseData.js
export const exercises = {
  'Bicep Curl': {
    primary: ['bicep_brachii_left', 'bicep_brachii_right'],
    secondary: ['brachialis_left', 'brachialis_right'],
  },
  'Bench Press': {
    primary: ['pectoralis_major_left', 'pectoralis_major_right'],
    secondary: ['deltoid_anterior_left', 'deltoid_anterior_right', 'triceps_brachii_left', 'triceps_brachii_right'],
  },
  // Add more exercises
};

Note: The muscle names here must match the name props of the <mesh> components in your AnatomyModel.js file.

Next, let's update AnatomyModel.js to handle the highlighting.

// src/AnatomyModel.js
import React, { useMemo } from 'react';
import { useGLTF } from '@react-three/drei';
import * as THREE from 'three';

// Define materials for different states
const defaultMaterial = new THREE.MeshStandardMaterial({ color: '#d3d3d3', roughness: 0.6 });
const primaryHighlightMaterial = new THREE.MeshStandardMaterial({ color: '#ff4136', emissive: '#ff4136', emissiveIntensity: 1 });
const secondaryHighlightMaterial = new THREE.MeshStandardMaterial({ color: '#ff851b', emissive: '#ff851b', emissiveIntensity: 0.5 });

export function AnatomyModel({ highlightedMuscles }) {
  const { nodes } = useGLTF('/models/anatomical_model.glb');
  
  // Memoize the nodes array to avoid re-creating on every render
  const modelNodes = useMemo(() => Object.values(nodes).filter(node => node.isMesh), [nodes]);

  return (
    <group dispose={null}>
      {modelNodes.map((node) => {
        let material = defaultMaterial;
        if (highlightedMuscles?.primary.includes(node.name)) {
          material = primaryHighlightMaterial;
        } else if (highlightedMuscles?.secondary.includes(node.name)) {
          material = secondaryHighlightMaterial;
        }
        
        return (
          <mesh
            key={node.uuid}
            name={node.name}
            geometry={node.geometry}
            material={material}
          />
        );
      })}
    </group>
  );
}

useGLTF.preload('/models/anatomical_model.glb');

Finally, let's add a UI to App.js to control the highlighted muscles.

// src/App.js
import React, { Suspense, useState } from 'react';
import { Canvas } from '@react-three/fiber';
import { OrbitControls } from '@react-three/drei';
import { AnatomyModel } from './AnatomyModel';
import { exercises } from './exerciseData';
import './App.css';

function App() {
  const [selectedExercise, setSelectedExercise] = useState('Bicep Curl');
  
  const highlightedMuscles = exercises[selectedExercise];

  return (
    <>
      <div className="controls">
        <label htmlFor="exercise-select">Choose an exercise:</label>
        <select
          id="exercise-select"
          value={selectedExercise}
          onChange={(e) => setSelectedExercise(e.target.value)}
        >
          {Object.keys(exercises).map(exercise => (
            <option key={exercise} value={exercise}>
              {exercise}
            </option>
          ))}
        </select>
      </div>
      <Canvas camera={{ position: [0, 0, 3], fov: 60 }}>
        {/* ... Canvas content ... */}
        <Suspense fallback={null}>
          <AnatomyModel highlightedMuscles={highlightedMuscles} />
        </Suspense>
        {/* ... */}
      </Canvas>
    </>
  );
}

export default App;

And some CSS for the controls:

/* src/App.css */
/* ... existing styles ... */

.controls {
  position: absolute;
  top: 20px;
  left: 20px;
  z-index: 10;
  background: rgba(0, 0, 0, 0.5);
  padding: 15px;
  border-radius: 10px;
  color: white;
}

How it works

1. State Management: We use a simple React useState hook in App.js to keep track of the currently selected exercise.
2. Data Flow: The selected exercise name is used to look up the corresponding primary and secondary muscle groups from our exerciseData object. This object is then passed as a prop (highlightedMuscles) to the AnatomyModel component.
3. Conditional Rendering: Inside AnatomyModel, we iterate over all the meshes from our loaded model. For each mesh, we check if its name is in the primary or secondary array of the highlightedMuscles prop.
4. Material Swapping: Based on the check, we apply one of three pre-defined materials. We use the emissive property to make the highlighted muscles "glow," giving a clear visual cue.

Now, when you select an exercise from the dropdown, the corresponding muscles on the 3D model should instantly light up! ✨

Here is the complete code for the main components for easy reference.

src/exerciseData.js

export const exercises = {
  'Bicep Curl': {
    primary: ['bicep_brachii_left', 'bicep_brachii_right'],
    secondary: ['brachialis_left', 'brachialis_right'],
  },
  'Bench Press': {
    primary: ['pectoralis_major_left', 'pectoralis_major_right'],
    secondary: ['deltoid_anterior_left', 'deltoid_anterior_right', 'triceps_brachii_left', 'triceps_brachii_right'],
  },
};

src/AnatomyModel.js

import React, { useMemo } from 'react';
import { useGLTF } from '@react-three/drei';
import * as THREE from 'three';

const defaultMaterial = new THREE.MeshStandardMaterial({ color: '#d3d3d3', roughness: 0.6 });
const primaryHighlightMaterial = new THREE.MeshStandardMaterial({ color: '#ff4136', emissive: '#ff4136', emissiveIntensity: 1 });
const secondaryHighlightMaterial = new THREE.MeshStandardMaterial({ color: '#ff851b', emissive: '#ff851b', emissiveIntensity: 0.5 });

export function AnatomyModel({ highlightedMuscles }) {
  const { nodes } = useGLTF('/models/anatomical_model.glb');
  
  const modelNodes = useMemo(() => Object.values(nodes).filter(node => node.isMesh), [nodes]);

  return (
    <group dispose={null}>
      {modelNodes.map((node) => {
        let material = defaultMaterial;
        if (highlightedMuscles?.primary.includes(node.name)) {
          material = primaryHighlightMaterial;
        } else if (highlightedMuscles?.secondary.includes(node.name)) {
          material = secondaryHighlightMaterial;
        }
        
        return (
          <mesh
            key={node.uuid}
            name={node.name}
            geometry={node.geometry}
            material={material}
          />
        );
      })}
    </group>
  );
}
useGLTF.preload('/models/anatomical_model.glb');

src/App.js

import React, { Suspense, useState } from 'react';
import { Canvas } from '@react-three/fiber';
import { OrbitControls } from '@react-three/drei';
import { AnatomyModel } from './AnatomyModel';
import { exercises } from './exerciseData';
import './App.css';

function App() {
  const [selectedExercise, setSelectedExercise] = useState('Bicep Curl');
  const highlightedMuscles = exercises[selectedExercise];

  return (
    <>
      <div className="controls">
        <label htmlFor="exercise-select">Choose an exercise:</label>
        <select id="exercise-select" value={selectedExercise} onChange={(e) => setSelectedExercise(e.target.value)}>
          {Object.keys(exercises).map(exercise => (<option key={exercise} value={exercise}>{exercise}</option>))}
        </select>
      </div>
      <div className="App">
        <Canvas camera={{ position: [0, 0, 3], fov: 60 }}>
          <ambientLight intensity={0.5} />
          <directionalLight position={[2, 5, 2]} intensity={1} />
          <Suspense fallback={null}>
            <AnatomyModel highlightedMuscles={highlightedMuscles} />
          </Suspense>
          <OrbitControls />
        </Canvas>
      </div>
    </>
  );
}

export default App;

Post-Processing for Better Highlighting

Our current approach of swapping materials is simple and effective. For a more advanced visual effect, you could use post-processing libraries like @react-three/postprocessing. The SelectiveBloom effect, for instance, can create a more realistic and visually striking glow around selected objects.

On-Model Interaction

Instead of a dropdown, you could allow users to click directly on a muscle to get more information. React Three Fiber supports pointer events like onClick and onPointerOver directly on meshes, making this a straightforward extension.

<mesh
  // ... other props
  onClick={(event) => console.log(`${event.object.name} clicked!`)}
  onPointerOver={() => setHovered(true)}
  onPointerOut={() => setHovered(false)}
/>

Congratulations! You've successfully built an interactive 3D anatomy model with React and React Three Fiber. This component provides a rich, educational experience that can significantly enhance any fitness or health-related application.

We've covered setting up a 3D scene, loading and parsing a complex GLTF model, managing state between a standard React UI and a 3D canvas, and implementing conditional highlighting logic. This project serves as a solid foundation that you can expand upon with more complex animations, post-processing effects, and deeper interactivity.

The possibilities are vast, from showing muscle contractions to overlaying anatomical information. I encourage you to take these concepts and build something truly unique.

• React Three Fiber Documentation: https://docs.pmnd.rs/react-three-fiber
• Drei (R3F Helpers) Documentation: https://github.com/pmndrs/drei
• Three.js Fundamentals: https://threejs.org/docs/