{"title": "Size Matters: Metric Visual Search Constraints from Monocular Metadata", "book": "Advances in Neural Information Processing Systems", "page_first": 622, "page_last": 630, "abstract": "Metric constraints are known to be highly discriminative for many objects, but if training is limited to data captured from a particular 3-D sensor the quantity of training data may be severly limited. In this paper, we show how a crucial aspect of 3-D information\u2013object and feature absolute size\u2013can be added to models learned from commonly available online imagery, without use of any 3-D sensing or re- construction at training time. Such models can be utilized at test time together with explicit 3-D sensing to perform robust search. Our model uses a \u201c2.1D\u201d local feature, which combines traditional appearance gradient statistics with an estimate of average absolute depth within the local window. We show how category size information can be obtained from online images by exploiting relatively unbiquitous metadata fields specifying camera intrinstics. We develop an efficient metric branch-and-bound algorithm for our search task, imposing 3-D size constraints as part of an optimal search for a set of features which indicate the presence of a category. Experiments on test scenes captured with a traditional stereo rig are shown, exploiting training data from from purely monocular sources with associated EXIF metadata.", "full_text": "Size Matters: Metric Visual Search Constraints from\n\nMonocular Metadata\n\nMario Fritz\n\nUC Berkeley EECS & ICSI\n\nKate Saenko\n\nUC Berkeley EECS & ICSI\n\nTrevor Darrell\n\nUC Berkeley EECS & ICSI\n\nAbstract\n\nMetric constraints are known to be highly discriminative for many objects, but\nif training is limited to data captured from a particular 3-D sensor the quantity of\ntraining data may be severly limited. In this paper, we show how a crucial aspect of\n3-D information\u2013object and feature absolute size\u2013can be added to models learned\nfrom commonly available online imagery, without use of any 3-D sensing or re-\nconstruction at training time. Such models can be utilized at test time together\nwith explicit 3-D sensing to perform robust search. Our model uses a \u201c2.1D\u201d\nlocal feature, which combines traditional appearance gradient statistics with an\nestimate of average absolute depth within the local window. We show how cate-\ngory size information can be obtained from online images by exploiting relatively\nunbiquitous metadata \ufb01elds specifying camera intrinstics. We develop an ef\ufb01-\ncient metric branch-and-bound algorithm for our search task, imposing 3-D size\nconstraints as part of an optimal search for a set of features which indicate the\npresence of a category. Experiments on test scenes captured with a traditional\nstereo rig are shown, exploiting training data from from purely monocular sources\nwith associated EXIF metadata.\n\n1\n\nIntroduction\n\nTwo themes dominate recent progress towards situated visual object recognition. Most signi\ufb01cantly,\nthe availability of large scale image databases and machine learning methods has driven perfor-\nmance: accuracy on many category detection tasks is a function of the quantity and quality of the\navailable training data. At the same time, when we consider situated recognition tasks, i.e., as\nperformed by robots, autonomous vehicles, and interactive physical devices (e.g., mobile phones),\nit is apparent that the variety and number of sensors is often what determines performance levels:\ne.g., the avaibility of 3-D sensing can signi\ufb01cantly improve performance on speci\ufb01c practical tasks,\nirrespective of the amount of training data. A rich variety of 3-D sensors are available on modern\nrobotic systems, yet the training data are few for most 3-D sensor regimes: the vast majority of\navailable online visual category data are from monocular sources and there are few databases of\nreal-world 3-D scans from which to train robust visual recognizers. In general it is, however, dif-\n\ufb01cult to reconcile these two trends: while one would like to use all available sensors at test time,\nthe paucity of 3D training data will mean few categories are well-de\ufb01ned with full 3-D models, and\ngeneralization performance to new categories which lack 3-D training data may be poor. In this\npaper, we propose a method to bridge this gap and extract features from typical 2D data sources that\ncan enhance recognition performance when 3D information is available at test time.\n\n1\n\n\fFigure 1: Recovery of object size from known camera intrinsics\n\nThe paradigm of recognition-by-local-features has been well established in the computer vision\nliterature in recent years. Existing recognition schemes are designed generally to be invariant to\nscale and size. Local shape descriptors based on 3-D sensing have been proposed (e.g., VIP [2]),\nas well as local 3-D descriptors (e.g., 3-D shape context and SIFT [4, 3]), but we are somewhat\nskeptical of the ability of even the most recent 3-D sensor systems to extract the detailed local\ngeometry required to reliably detect and describe local 3-D shapes on real world objects.\nInstead of extracting full 3D local features, we propose a \u201c2.1D\u201d local feature model which augments\na traditional 2D local feature (SIFT, GLOH, SURF, etc.) with an estimate of the depth and 3-D size\nof an observed patch. Such features could distinguish, for example, the two different keypad patterns\non a mobile device keyboard vs. on a full-size computer keyboard; while the keys might look locally\nsimilar, the absolute patch size would be highly distinctive. We focus on the recognition of real-\nworld objects when additional sensors are available at test time, and show how 2.1D information\ncan be extracted from monocular metadata already present in many online images. Our model\nincludes both a representation of the absolute size of local features, and of the overall dimension\nof categories. We recover the depth and size of the local features, and thus of the bounding box of\na detected object in 3-D. Ef\ufb01cient search is an important goal, and we show a novel extension to\nmulti-class branch-and-bound search using explicit metric 3-D constraints.\n\n2 Recognition with \u201c2.1D\u201d features\n\nThe crux of our method is the inference and exploitation of size information; we show that we\ncan obtain such measurements from non-traditional sources that do not presume a 3-D scanner at\ntraining time, nor rely on multi-view reconstruction / structure-from-motion methods. We instead\nexploit cues that are readily available in many monocular camera images.1 We are not interested\nin reconstructing the object surface, and only estimate the absolute size of local patches, and the\nstatistics of the bounding box of instances in the category; from these quantities we can infer the\ncategory size.\nWe adopt a local-feature based recognition model and augment it with metric size information.\nWhile there are several possible local feature recognition schemes based on sets of such local fea-\ntures, we focus on the Naive Bayes nearest-neighbor model of [1] because of its simplicity and\ngood empirical results. We assume one or more common local feature descriptors (and associated\ndetectors or dense sampling grids): SIFT, SURF, GLOH, MSER. Our emphasis in this paper is on\n\n1There are a number of general paradigms by which estimates of object size can be extracted from a 2D\nimage data source, e.g., regression from scene context [6]), or inference of depth-from-a-single-image [7, 11,\n16]. In addition to such schemes, text associated with the training images extracted from internet merchants\n(e.g., Amazon, eBay) typically explicitly de\ufb01nes a bounding volume for the object. While all these are of\ninterest, we consider here only the use of methods based implicitly on depth-from-focus (e.g., [8]), present\nas camera intrinsics stored as metadata in the JPEG EXIF \ufb01le format.\nImages collected by many modern\nconsumer-grade digital SLR cameras automatically store absolute distance-to-subject as metadata in the JPEG\nimage.\n\n2\n\n\fFigure 2: Illustration of metric object size derived from image metadata stored in EXIF \ufb01elds on an\nimage downloaded from Flickr.com. Absolute size is estimated by projecting bounding box of local\nfeatures on object into 3-D using EXIF camera intrinsics stored in image \ufb01le format.\n\nimproving the accuracy of recognizing categories that are at least approximately well modeled with\nsuch-local feature schemes; size information alone cannot help recognize a category that does not\nrepeatably and reliably produce such features.\n\n2.1 Metric object size from monocular metadata\n\nAbsolute pixel size can be infered using a planar object approximation and depth from focus cues.\nToday\u2019s digital cameras supplement the image data with rich meta-data provided in the EXIF format.\nEXIF stores a wide range of intrinsic camera parameters, which often include the focus distance as\nan explicit parameter (in some cameras it is not provided directly, but can be estimated from other\nprovided parameters). This gives us a workable approximation of the depth of the object, assuming\nit is in focus in the scene: with a pinhole camera model, we can derive the metric size of a pixel\nf r ,\nin the scene given these assumptions. Using simple trigonometry, the metric pixel size is \u03c1 = sd\nwhere s is the sensor width, d is the focus distance, f is the focal length, and r is the horizontal\nresolution of the sensor.\nAs shown in Figure 2, this method provides a size estimate reference for the visual observation based\non images commonly available on the internet, e.g., Flickr.com. A bounding box can either be esti-\nmated from the feature locations, given an uncluttered background, or provided by manual labeling\nor by an object discovery technique which clusters local features to discover the segmentation of the\ntraining data.\n\n2.2 Naive Bayes estimation of discriminative feature weights\n\nOur object model is based on a bag-of-words model where an object is encoded by a set of visual\nfeatures xi \u2208 X within the circumscribing bounding box. Our size-constrained learning scheme\nis applicable to a range of recognition methods; for simplicity we adopt a simple but ef\ufb01cient non-\nparametric naive Bayes scheme. We denote object appearance with p(X|C); following [1], this\ndensity can be captured and modeled using Parzen window density estimates:\n\n3\n\n\fFigure 3: Metric object size for ten different categories derived from camera metadata. Bold symbols\ndepict ground truth obtained by direct physical measurement of category instance.\n\nN(cid:88)\n\nj=1\n\nK(x \u2212 xC\nj ),\n\n\u02c6p(x|C) =\n\n1\nN\n\n(1)\n\nwhere K(.) is a Gaussian kernel.\nWe extend this model in a discriminative fashion similar to [18]. We compute the detection score for\na given bounding box from the log-likelihood ratio computed based on the kernel density estimate\nfrom above. Assuming independence of the features, the class spec\ufb01c probabilities are factorized to\nobtain a sum of individual feature contributions:\n\np(X|C)\np(X| \u00afC)\n\nlog\n\n(cid:81)\n(cid:81)\n\n(cid:88)\n\ni p(xi|C)\ni p(xi| \u00afC)\nlog(p(xi|C) \u2212 log(p(xi| \u00afC))\n\n= log\n\n=\n\n(2)\n\n(3)\n\ni\n\nAs shown in [1], an approximate density based only on the nearest neighbor is accurate for many\nrecognition tasks. This further simpli\ufb01es the computation and approximates the class speci\ufb01c feature\nprobabilities by:\n\nlog p(xi|C) \u2248 ||xi \u2212 N NC(xi)||2,\n\n(4)\nwhere N NC(xi) represents the distance of data point xi to the nearest example in the training data\nof class C.\nIn the multi-class case, each feature xi is compared to the nearest neighbors in the\ntraining examples of each class, N N \u00afC(xi) can be simply obtained as the minimum of all retrieved\nnearest neighbors except those in C.\n\n3 Ef\ufb01cient search with absolute size\n\nRecently, a class of algorithms for ef\ufb01cient detection based on local features has been proposed\n[19, 20, 21]; these search for the highest-scoring bounding box given the observed features X and a\n\n4\n\n\fscoring function f using an ef\ufb01cient branch-and-bound scheme. These methods can be formulated\nas an optimization b = arg maxb f (b), where b = (x1, y1, x2, y2) is a bounding box. The core idea\nis to structure the search space using a search tree. The top node contains the set of all possible\nbounding boxes. The child nodes contain splits of the set of bounding boxes in the parent node. The\nleafs contain single bounding boxes. If it is possible to derive lower and upper bounds for rectangle\nsets at the nodes, a branch and bound technique can be applied to quickly prune nodes if its upper\nbound is lower than the lower bound of a previously visited node.\nBounds can be easily computed for bag-of-words representations, which have been previously used\nin this context for object detection. Each feature has a learned weight wj, wherefore the score\nfunction f reads:\n\n(cid:88)\n\nj\u2208T (b)\n\nf (r) =\n\nwj,\n\n(5)\n\nwhere T (b) is the set of all features contained in the bounding box b.\nWhile previous approaches have derived the feature weight from SVM training, we propose to use\nlikelihood ratios which are derived in a non-parametric fashion.\nWe further extend this method to search for objects in 3d. Our bounding box hypotheses b =\n(x1, y1, z1, x2, y2, z2) are de\ufb01ned explicitly in 3d and indicate the actual spatial relation of objects\nin the scene.\nWe employ a constraint factor S(b) to the objective that indicates if a bounding box has a valid size\ngiven a particular class or not:\n\n(cid:88)\n\nj\u2208T (b)\n\nf (r) =\n\nwjS(b)\n\n(6)\n\nS(b) = 1 is a basic rectangle function that takes the value 1 for valid bounding boxes and 0 other-\nwise.\nMost importantly, bounds over bounding box sets can still be ef\ufb01ciently computed. As long as the\nbounding box set at a given node in the search tree contains at least one bounding box of valid size,\nthe score is unaffected. When there is no valid rectangle left, the score evaluates to zero and that\nnode as well as the associated sub space of the search problem gets pruned.\nAt test time, it is anticipated that 3D observations are directly available via LIDAR scans or active\nor passive stereo estimation. Given these measurements, we constrain the search to leverage the\nmetric information acquired at training time. The depth for each feature in the image at test time\nallows us to infer their 3D location in the test scene. We can thus extend ef\ufb01cient multi-class branch-\nand-bound search to operate in metric 3D space under the constraints imposed by our knowledge of\nmetric patch size and metric object size.\nWe also make use of the proposed multi-class branch-and-bound scheme as proposed in [20]. We\nnot only split bounding box sets along dimensions, but also split the set of object classes. This leads\nto a simultaneous search scheme for multiple classes.\n\n4 Related Work\n\nMany methods have been proposed to deal with the problem of establishing feature correspondence\nacross varying image scales. Lowe et. al. proposed to up/downsample an image at multiple scales\nand identify the characterstic scale for each image patch [9]. A histogram of edge orientations is\ncomputed for each patch scaled to its characteristic scale in order to obtain a scale-invariant visual\ndescriptor. [10] identi\ufb01es scale invariant regions by interatively expanding consistent regions with\nan increasing intensity threshold until they become \u201dstable\u201d. The size of the stable region is the\ncharactersitic scale for the feature. With both methods, a feature in one image can be mapped to the\nsame characteristic scale a feature in another image. Since both features are mapped to the same\n\n5\n\n\fscale, an \u201dapple-to-apple\u201d comparison can be performed. In contrast, our method does not require\nsuch a mapping. Instead, it determines the metric size of any image patch and uses it to compare\ntwo features directly.\nThere have been several works on estimating depth from single images. Some very early work\nestimated depth from the degree of the defocus of edges [8]. [6] describes a method to infer scene\ndepth from structure baesd on global and local histograms of Gabor \ufb01lter responses for indoor and\noutdoor scenes. [11] describes a supervised Markov Random Field method to predict the depth from\nlocal and global features for outdoor images. In our work, we focus on indoor of\ufb01ce scenes with\n\ufb01ner granularity. Hardware-based methods for obtaining 3D information from monocular images\ninclude modifying the structure of a conventional camera to enable it to capture 3D geometry. For\nexample, [12] introduces the coded aperture technique by inserting a patterned occluder within the\naperture of the camera lenses. Images captured by such a camera exhibit depth-dependent patterns\nfrom which a layered depth map can be extracted.\nMost methods based on visual feature quantization learn their codebooks using invariant features.\nHowever, the scale of each code word is lost after each image patch is normalized to its invariant re-\ngion. Thus, it is possible for two features to match because they happen to look similar, even though\nin the physical world they actually have two different sizes. For example, an eye of a dinosaur may\nbe confused with an eye of a \ufb01sh, because their size difference is lost once they are embedded into\nthe visual code book. There have been some proposals to deal with this problem. For example,\n[13] records the relative position of the object center in the codebook, and at test time each code-\nbook word votes for the possible object center at multiple scales. Moreover, [14] explicitly put the\norientation and scale of each feature in the codebook, so that object center location can be inferred\ndirectly. However, these works treat orientation and scale as independent of the feature descriptor\nand use them to post-verify whether a feature found to be consistent in terms of the appearance\ndesciptor would also be consistent in terms of scale. In contrast, our work directly embeds the scale\nattribute into the visual descriptor. A visual word would be matched only if its size is right. In other\nwords, the visual apperance and the scale are matched simulaneously in our codebook.\nDepth information has been used to improve the performance of various image processing tasks,\nsuch as video retrieval, object instance detection, 3D scene recognition, and vehicle navigation.\nFor example, [15] used depth feature for video retrieval, extracting depth from monocular video\nsequences by exploiting the motion parallax of the objects in the video. [16] developed an intergrated\nprobablistic model for apperance and 3D geometry of object categories. However, their method\ndoes not expliclty assign physical size to each image patch and needs to provide scale-invariance by\nexplictly calculating the perspective projection of objects in different 3D poposes. In contrast, our\nmethod can infer the real-world sizes of features and can establish feature correspondences at their\ntrue physical scale. [17] proposed a way to use depth estimation for real-time obstacle detection\nfrom a monocular video stream in a vehicle navigation scenario. Their method estimates scene\ndepth from the scaling of supervised image regions and generates obstacle hypotheses from these\ndepth estimates.\n\n5 Experiments\n\nIn the experiments we show how to improve performance of visual object classi\ufb01ers by leveraging\nricher sensor modalities deployed at test time. We analyze how the different proposed means of\nputting visual recognition in metric context improves detection performance.\n\n5.1 Data\n\nFor training we explore the camera-based metadata scheme described above, where we derive the\nmetric pixel size from EXIF data. We downloaded 38 images of 10 object categories taken with\na consumer grade dSLR that stores relevant EXIF \ufb01elds (e.g., Nikon D90). For test data we have\ncollected 34 scenes in our laboratory of varying complexity containing 120 object instances in of\ufb01ces\nand and a kitchen. Considerable levels of clutter, lighting and occlusion are present in the test set.\nStereo depth observations using a calibrated camera rig are obtained with test imagery, providing an\nestimate of the 3-D depth of each feature point at test time.\n\n6\n\n\fFigure 4: Example detections.\n\n7\n\n\fobject\n\nbike helmet\nbody wash\n\njuice\nkleenex\n\nmug\npasta\nphone\npringles\ntoothpaste\nvitamins\naverage\n\nbaseline\n\n89.0\n3.3\n76.0\n60.0\n0.0\n36.3\n80.0\n45.8\n20.0\n0.0\n41.0\n\n2.1D\n99.1\n80.0\n100.0\n76.53\n24.63\n65.6\n65.7\n94.3\n100.0\n60.0\n76.59\n\nTable 1: Average precision for several categories for baseline 2-D branch and bound search and our\n2.1D method.\n\n5.2 Evaluation\n\nWe start with a baseline, which uses the plain branch and bound detection scheme and 2D features.\nWe then experiment with augment the representation to 2.1D, adding 3D location to the interest\npoints, as well as employing the metric size constraint.\nTable 1 shows the average precision for each category for baseline 2-D branch and bound search\nand our 2.1D method. Adding the metric object constraints (second column) improves the results\nsigni\ufb01cantly. As illustrated in Figure 4, our 2.1D representation allows grouping in 3-D and provides\nimproved occlusion handling. We see that the baseline branch-and-bound performs poorly on this\ndata set and is not capable of localizing two of the items at all. For the training data available\nfor these categories the local evidence was apparently not strong enough to support this detection\nscheme, but with size constraints performance improved signi\ufb01cantly.\n\n6 Conclusion\n\nProgress on large scale systems for visual categorization has been driven by the abundance of train-\ning data available from the web. Much richer and potentially more discriminative measurements can\nbe acquired and leveraged by additional sensor modalities, e.g. 3D measurements from stereo or\nlidar, typically found on contemporary robotic platforms, but there is rarely suf\ufb01cient training data\nto learn robust models using these sensors. In order to reconcile these two trends, we developed a\nmethod for appearance-based visual recognition in metric context, exploiting camera-based meta-\ndata to obtain size information regarding a category and local feature models that can be exploited\nusing 3-D sensors at test time.\nWe believe that \u201csize matters\u201d, and that the most informative and robust aspect of 3-D information is\ndimensional. We augmented local feature-based visual models with a \u201c2.1D\u201d object representation\nby introducing the notion of a metric patch size. Scene context from 3-D sensing and category-level\ndimension estimates provide additional cues to limit search. We presented a fast, multi-class detec-\ntion scheme based on a metric branch-and-bound formulation. While our method was demonstrated\nonly on simple 2-D SURF features, we belive these methods will be applicable as well to multi-\nkernel schemes with additional feature modalities, as well as object level desriptors (e.g., HOG,\nLatentSVM).\n\nAcknowledgements. This work was supported in part by TOYOTA and a Feodor Lynen Fellow-\nship granted by the Alexander von Humboldt Foundation.\n\n8\n\n\fReferences\n[1] O. Boiman, E. Shechtman, and M. Irani, In defense of Nearest-Neighbor based image classi\ufb01-\n\ncation, In Proceedings of Computer Vision and Pattern Recognition, 2008.\n\n[2] C. Wu, B. Clipp, X. Li, J.-M. Frahm, and M. Pollefeys, 3D model matching with Viewpoint-\n\nInvariant Patches (VIP), In Proceedings of Computer Vision and Pattern Recognition, 2008.\n\n[3] P. Scovanner, S. Ali, M. Shah, A 3-dimensional SIFT descriptor and its application to action\n\nrecognition, In Proceedings of the 15th international conference on Multimedia, 2007.\n\n[4] M. Kortgen, G. J. Park, M. Novotni, R. Klein, 3D Shape Matching with 3D Shape Contexts, In\n\nthe 7th Central European Seminar on Computer Graphics, 2003.\n\n[5] A. Frome, D. Huber, R. Kolluri, T. Bulow, and J. Malik. Recognizing objects in range data using\nregional point descriptors, In Proceedings of the 8th European Conference on Computer Vision,\n2004.\n\n[6] A. Oliva, and A. Torralba, Building the Gist of a Scene: The Role of Global Image Features in\n\nRecognition, In Visual Perception, Progress in Brain Research, vol 155, 2006.\n\n[7] D. Hoiem, A. Efros, M. Hebert, Geometric Context from a Single Image, In Proceedings of the\n\nTenth IEEE International Conference on Computer Vision, 2005.\n\n[8] T. Darrell and K. Wohn, Pyramid based depth from focus, In Proceedings of Computer Vision\n\nand Pattern Recognition, 1988.\n\n[9] D. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of\n\nComputer Vision, 2004.\n\n[10] J. Matas, O. Chum, and M. Urban, and T. Pajdla, Robust wide baseline stereo from maximally\n\nstable extremal regions. In British Machine Vision Conference, 2002.\n\n[11] A. Saxena, M. Sun, A. Y. Ng, Make3D: Learning 3-D Scene Structure from a Single Still\n\nImage, In IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2008.\n\n[12] A. Levin, R. Fergus, F. Durand, Fr\u00b4edo, and W.T. Freeman, Image and depth from a conven-\n\ntional camera with a coded aperture, ACM Transactions on Graphics, 2007.\n\n[13] Bastian Leibe and Ales Leonardis and Bernt Schiele, Combined Object Categorization and\nSegmentation With An Implicit Shape Model In ECCV workshop on statistical learning in com-\nputer vision, 2004\n\n[14] Krystian Mikolajczyk and Cordelia Schmid, A Performance Evaluation of Local Descriptors,\n\nIn PAMI, 2005.\n\n[15] R. Ewerth, M. Schwalb, Martin, and B. Freisleben, Using depth features to retrieve monoc-\nular video shots, In Proceedings of the 6th ACM international conference on image and video\nretrieval, 2007.\n\n[16] E. Sudderth, A. Torralba, W. T. Freeman, and A. Wilsky, Depth from Familiar Objects: A\nHierarchical Model for 3D Scenes, In Proceedings of Computer Vision and Pattern Recognition,\n2006.\n\n[17] A. Wedel, U. Franke, J. Klappstein, T. Brox, and D. Cremers, Realtime Depth Estimation and\n\nObstacle Detection from Monocular Video, DAGM-Symposium, 2006.\n\n[18] Junsong Yuan, Zicheng Liu and Ying Wu, Discriminative Subvolume Search for Ef\ufb01cient Ac-\n\ntion Detection, In Proceedings of Computer Vision and Pattern Recognition, 2009.\n\n[19] Christoph H. Lampert and Matthew B. Blaschko and Thomas Hofmann, Ef\ufb01cient Subwindow\nSearch: A Branch and Bound Framework for Object Localization, In Transactions on Pattern\nAnalysis and Machine Intelligence (PAMI), 2009.\n\n[20] Tom Yeh, John Lee and Trevor Darrell, Fast Concurrent Object Localization and Recognition,\n\nIn CVPR 2009.\n\n[21] Junsong Yuan and Zicheng Liu and Ying Wu, Discriminative Subvolume Search for Ef\ufb01cient\n\nAction Detection, In CVPR 2009.\n\n9\n\n\f", "award": [], "sourceid": 1221, "authors": [{"given_name": "Mario", "family_name": "Fritz", "institution": null}, {"given_name": "Kate", "family_name": "Saenko", "institution": null}, {"given_name": "Trevor", "family_name": "Darrell", "institution": null}]}