The COCO keypoint format: 17 keypoints, order, and skeleton.

Everything you need to read or write COCO person-keypoint annotations: the exact keypoint order, the official skeleton pairs, what the visibility flags mean, and worked JSON examples. Bookmark-grade; no scrolling through a tutorial to find the list.

Summary

COCO person keypoints are 17 points in a fixed order, from nose to right ankle. Each annotation stores them as one flat list of x, y, v triplets, 51 numbers per person. The visibility flag v is 0 when the point is not labeled, 1 when it is labeled but hidden, and 2 when it is labeled and visible. The skeleton is 19 pairs of keypoint indices, counted from 1.

Looking for keypoint datasets rather than the format? See Keypoint datasets.

Background

What this format is, and where it comes from

COCO (Common Objects in Context) is the large-scale image dataset Microsoft released in 2014; its 2016 keypoint challenge labeled roughly 250,000 people with the same 17 body points, and the JSON layout of those annotations became the standard interchange format for 2D pose. Nearly every pose library reads or writes it, from pycocotools to OpenPose, Detectron2, MMPose, and YOLO. The layout is not person-specific (a category can define any names and skeleton), but the person schema below is what “COCO keypoints” means unqualified.

The 17 keypoints

Keypoint order

The keypoints array is 0-based, but the skeleton pairs reference 1-based ids, the most common off-by-one in pose tooling. Both are listed. “Left” always means the subject’s left, which appears on the viewer’s right in a front-facing image.

Array index (0-based)	Skeleton id (1-based)	Name	Side
0	1	nose	–
1	2	left_eye	left
2	3	right_eye	right
3	4	left_ear	left
4	5	right_ear	right
5	6	left_shoulder	left
6	7	right_shoulder	right
7	8	left_elbow	left
8	9	right_elbow	right
9	10	left_wrist	left
10	11	right_wrist	right
11	12	left_hip	left
12	13	right_hip	right
13	14	left_knee	left
14	15	right_knee	right
15	16	left_ankle	left
16	17	right_ankle	right

Numbered with 1-based skeleton ids

The skeleton

The official 19 skeleton pairs

Each pair connects two keypoints by their 1-based ids. This is the canonical list from the COCO annotations, verbatim:

"skeleton": [
  [16,14], [14,12], [17,15], [15,13], [12,13], [6,12], [7,13], [6,7], [6,8], [7,9], [8,10], [9,11], [2,3], [1,2], [1,3], [2,4], [3,5], [4,6], [5,7]
]

Visibility flags

What v = 0, 1, 2 mean

Keypoints are stored as flat [x1, y1, v1, x2, y2, v2, …] triplets, 51 numbers for a person. The third value of each triplet is the visibility flag:

v = 0

Not labeled

x and y are 0; the point was not annotated at all.

v = 1

Labeled, not visible

The point has coordinates but is occluded (e.g. a hip under a coat).

v = 2

Labeled and visible

The point is annotated and visible in the image.

num_keypoints on the annotation counts the labeled points: those with v > 0. New to keypoint annotation itself? Start with the practical guide; looking for data in this format, see keypoint datasets.

Putting it together

The category and annotation JSON

The category declares the names and skeleton; each annotation carries the triplets. A minimal, valid pair:

// categories[]
{
  "id": 1,
  "name": "person",
  "supercategory": "person",
  "keypoints": [
    "nose", "left_eye", "right_eye",
    "left_ear", "right_ear",
    "left_shoulder", "right_shoulder",
    "left_elbow", "right_elbow",
    "left_wrist", "right_wrist",
    "left_hip", "right_hip",
    "left_knee", "right_knee",
    "left_ankle", "right_ankle"
  ],
  "skeleton": [
    [16,14],[14,12],[17,15],[15,13],
    [12,13],[6,12],[7,13],[6,7],
    [6,8],[7,9],[8,10],[9,11],
    [2,3],[1,2],[1,3],[2,4],
    [3,5],[4,6],[5,7]
  ]
}

// annotations[]
{
  "id": 42,
  "image_id": 7,
  "category_id": 1,
  "num_keypoints": 3,
  "keypoints": [
    412, 143, 2,   // nose: visible
    431, 128, 2,   // left_eye: visible
    398, 129, 1,   // right_eye: occluded
    0,   0,   0,   // left_ear: not labeled
    // … one [x, y, v] triplet per
    // keypoint, 17 in total
  ],
  "bbox": [372, 94, 118, 340],
  "iscrowd": 0,
  "area": 40120
}

COCO keypoints FAQ

Common questions#

What are the 17 COCO keypoints in order?

nose, left_eye, right_eye, left_ear, right_ear, left_shoulder, right_shoulder, left_elbow, right_elbow, left_wrist, right_wrist, left_hip, right_hip, left_knee, right_knee, left_ankle, right_ankle.

What do COCO visibility flags 0, 1 and 2 mean?

0: not labeled (x and y are 0). 1: labeled but not visible. 2: labeled and visible.

What is num_keypoints in a COCO annotation?

The number of keypoints with v greater than 0, so labeled ones, visible or not.

Are COCO skeleton indices 0-based or 1-based?

1-based. The skeleton pair [16, 14] joins left_ankle and left_knee.

Annotate in this format

Skip the conversion scripts.

DataTorch speaks this format natively: define the keypoints and skeleton on a label, annotate point-by-point in the browser with a live skeleton overlay, and import or export COCO keypoints, visibility flags included.

See the keypoint tool Start free

Tools and community for image annotation: label, review, and share datasets, from everyday photos to the imagery only specialists can read.