{"id":51,"date":"2025-03-05T13:52:39","date_gmt":"2025-03-05T13:52:39","guid":{"rendered":"https:\/\/michaelmaynord.net\/?p=51"},"modified":"2025-03-05T13:52:39","modified_gmt":"2025-03-05T13:52:39","slug":"forecasting-action-through-contact-representations-from-first-person-video","status":"publish","type":"post","link":"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/","title":{"rendered":"Forecasting action through contact representations from first person video"},"content":{"rendered":"\n<p>Human actions involving hand manipulations are structured according to the making and breaking of hand-object contact, and human visual understanding of action is reliant on anticipation of contact as is demonstrated by pioneering work in cognitive science. Taking inspiration from this, we introduce representations and models centered on contact, which we then use in action prediction and anticipation. We annotate a subset of the EPIC Kitchens dataset to include time-to-contact between hands and objects, as well as segmentations of hands and objects. Using these annotations we train the <em>Anticipation Module<\/em>, a module producing <em>Contact Anticipation Maps<\/em> and <em>Next Active Object Segmentations<\/em> &#8211; novel low-level representations providing temporal and spatial characteristics of anticipated near future action. On top of the Anticipation Module we apply <em>Egocentric Object Manipulation Graphs<\/em> (Ego-OMG).<\/p>\n\n\n\n<p>Read more here: <a href=\"https:\/\/arxiv.org\/pdf\/2102.00649\">https:\/\/arxiv.org\/pdf\/2102.00649<\/a><\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Human actions involving hand manipulations are structured according to the making and breaking of hand-object contact, and human visual understanding of action is reliant on anticipation of contact as is&#8230; <\/p>\n","protected":false},"author":1,"featured_media":52,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-51","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Forecasting action through contact representations from first person video - Michael Maynord<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Forecasting action through contact representations from first person video - Michael Maynord\" \/>\n<meta property=\"og:description\" content=\"Human actions involving hand manipulations are structured according to the making and breaking of hand-object contact, and human visual understanding of action is reliant on anticipation of contact as is...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/\" \/>\n<meta property=\"og:site_name\" content=\"Michael Maynord\" \/>\n<meta property=\"article:published_time\" content=\"2025-03-05T13:52:39+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2025\/03\/Forecasting-Action.png\" \/>\n\t<meta property=\"og:image:width\" content=\"418\" \/>\n\t<meta property=\"og:image:height\" content=\"294\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/\",\"url\":\"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/\",\"name\":\"Forecasting action through contact representations from first person video - Michael Maynord\",\"isPartOf\":{\"@id\":\"https:\/\/michaelmaynord.net\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2025\/03\/Forecasting-Action.png\",\"datePublished\":\"2025-03-05T13:52:39+00:00\",\"author\":{\"@id\":\"https:\/\/michaelmaynord.net\/#\/schema\/person\/4032622d57ec607730c1f229bb9fe96c\"},\"breadcrumb\":{\"@id\":\"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/#primaryimage\",\"url\":\"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2025\/03\/Forecasting-Action.png\",\"contentUrl\":\"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2025\/03\/Forecasting-Action.png\",\"width\":418,\"height\":294,\"caption\":\"Forecasting Action\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/michaelmaynord.net\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Forecasting action through contact representations from first person video\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/michaelmaynord.net\/#website\",\"url\":\"https:\/\/michaelmaynord.net\/\",\"name\":\"Michael Maynord\",\"description\":\"Research Scientist\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/michaelmaynord.net\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/michaelmaynord.net\/#\/schema\/person\/4032622d57ec607730c1f229bb9fe96c\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/michaelmaynord.net\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2026\/03\/cropped-Transparent-Avatar-96x96.png\",\"contentUrl\":\"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2026\/03\/cropped-Transparent-Avatar-96x96.png\",\"caption\":\"admin\"},\"sameAs\":[\"https:\/\/michaelmaynord.net\"],\"url\":\"https:\/\/michaelmaynord.net\/index.php\/author\/robert_4gn3jyiv\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Forecasting action through contact representations from first person video - Michael Maynord","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/","og_locale":"en_US","og_type":"article","og_title":"Forecasting action through contact representations from first person video - Michael Maynord","og_description":"Human actions involving hand manipulations are structured according to the making and breaking of hand-object contact, and human visual understanding of action is reliant on anticipation of contact as is...","og_url":"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/","og_site_name":"Michael Maynord","article_published_time":"2025-03-05T13:52:39+00:00","og_image":[{"width":418,"height":294,"url":"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2025\/03\/Forecasting-Action.png","type":"image\/png"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/","url":"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/","name":"Forecasting action through contact representations from first person video - Michael Maynord","isPartOf":{"@id":"https:\/\/michaelmaynord.net\/#website"},"primaryImageOfPage":{"@id":"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/#primaryimage"},"image":{"@id":"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/#primaryimage"},"thumbnailUrl":"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2025\/03\/Forecasting-Action.png","datePublished":"2025-03-05T13:52:39+00:00","author":{"@id":"https:\/\/michaelmaynord.net\/#\/schema\/person\/4032622d57ec607730c1f229bb9fe96c"},"breadcrumb":{"@id":"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/#primaryimage","url":"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2025\/03\/Forecasting-Action.png","contentUrl":"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2025\/03\/Forecasting-Action.png","width":418,"height":294,"caption":"Forecasting Action"},{"@type":"BreadcrumbList","@id":"https:\/\/michaelmaynord.net\/index.php\/2025\/03\/05\/forecasting-action-through-contact-representations-from-first-person-video\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/michaelmaynord.net\/"},{"@type":"ListItem","position":2,"name":"Forecasting action through contact representations from first person video"}]},{"@type":"WebSite","@id":"https:\/\/michaelmaynord.net\/#website","url":"https:\/\/michaelmaynord.net\/","name":"Michael Maynord","description":"Research Scientist","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/michaelmaynord.net\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/michaelmaynord.net\/#\/schema\/person\/4032622d57ec607730c1f229bb9fe96c","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/michaelmaynord.net\/#\/schema\/person\/image\/","url":"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2026\/03\/cropped-Transparent-Avatar-96x96.png","contentUrl":"https:\/\/michaelmaynord.net\/wp-content\/uploads\/2026\/03\/cropped-Transparent-Avatar-96x96.png","caption":"admin"},"sameAs":["https:\/\/michaelmaynord.net"],"url":"https:\/\/michaelmaynord.net\/index.php\/author\/robert_4gn3jyiv\/"}]}},"_links":{"self":[{"href":"https:\/\/michaelmaynord.net\/index.php\/wp-json\/wp\/v2\/posts\/51","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/michaelmaynord.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/michaelmaynord.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/michaelmaynord.net\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/michaelmaynord.net\/index.php\/wp-json\/wp\/v2\/comments?post=51"}],"version-history":[{"count":1,"href":"https:\/\/michaelmaynord.net\/index.php\/wp-json\/wp\/v2\/posts\/51\/revisions"}],"predecessor-version":[{"id":53,"href":"https:\/\/michaelmaynord.net\/index.php\/wp-json\/wp\/v2\/posts\/51\/revisions\/53"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/michaelmaynord.net\/index.php\/wp-json\/wp\/v2\/media\/52"}],"wp:attachment":[{"href":"https:\/\/michaelmaynord.net\/index.php\/wp-json\/wp\/v2\/media?parent=51"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/michaelmaynord.net\/index.php\/wp-json\/wp\/v2\/categories?post=51"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/michaelmaynord.net\/index.php\/wp-json\/wp\/v2\/tags?post=51"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}