Abstract
A pressing question when designing intelligent autonomous systems is how to integrate the various subsystems concerned with complementary tasks. Robotic vision must provide task relevant information about the environment and the objects in it to various planning related modules. In most implementations of the traditional Perception–Cognition–Action paradigm these tasks are treated as quasi-independent modules that function as black boxes for each other. Often these subsystems are running in completely different frameworks, with a thin communication interface or middle-ware between them. While each subproblem poses specific requirements that can make fusing them more challenging, perception can benefit tremendously from a tight collaboration with cognition. In the following, a common framework for cognitive perception, based on the principle of unstructured information management (UIM) will be presented, called RoboSherlock. UIM has proven itself to be a powerful paradigm for scaling intelligent information and question answering systems towards real-world complexity. Complexity in UIM is handled by identifying (or hypothesizing) pieces of structured information by applying ensembles of experts for annotating information pieces, and by testing and integrating these isolated annotations into a comprehensive interpretation. RoboSherlock is an open source software framework for unstructured information processing in robot perception that demonstrates the potential of the paradigm for real-world scene perception.
Former and current members of Prof. Beetz’s research group sorted in alphabetical order.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
E.g. pancakes: http://youtu.be/4usoE981e7I popcorn/sandwich: http://youtu.be/DTaeWITW1kI and breakfast: https://youtu.be/gbIDPqb_2iM.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
For a complete list of available feature descriptors see www.pointclouds.org.
- 9.
- 10.
In contrast, data flow in e.g. ROS is very rigid, and message channel (“topic”) connectivity encodes semantic meaning of data. Addition or modification of information to messages can be cumbersome.
References
Aydemir A, Sjöö K, Folkesson J, Pronobis A, Jensfelt P (2011) Search in the real world: active visual object search based on spatial relations. In: IEEE international conference on robotics and automation (ICRA), 2011. IEEE, pp 2818–2824
Beetz M, Mösenlechner L, Tenorth M (2010) Cram—a cognitive robot abstract machine for everyday manipulation in human environments. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems. Taipei, Taiwan, pp 1012–1017
Beetz M, Balint-Benczedi F, Blodow N, Nyga D, Wiedemeyer T, Marton ZC (2015) RoboSherlock: unstructured information processing for robot perception. In: IEEE international conference on robotics and automation (ICRA), Seattle, Washington, USA, nominations for Best Conference Paper Award and Best Service Robotics Paper Award
Blodow N (2014) Managing belief states for service robots—dynamic scene perception and spatio-temporal memory. Ph.D. thesis, Technische Universität München. http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:91-diss-20140623-1174074-0-4
Blodow N, Jain D, Marton ZC, Beetz M (2010) Perception and probabilistic anchoring for dynamic world state logging. In: 10th IEEE-RAS international conference on humanoid robots. Nashville, TN, USA, pp 160–166
Blodow N, Marton ZC, Pangercic D, Rühr T, Tenorth M, Beetz M (2011) Inferring generalized pick-and-place tasks from pointing gestures. In: IEEE international conference on robotics and automation (ICRA), workshop on semantic perception, map** and exploration
Bohren J, Rusu RB, Jones EG, Marder-Eppstein E, Pantofaru C, Wise M, Mosenlechner L, Meeussen W, Holzer S (2011) Towards autonomous robotic butlers: lessons learned with the PR2. In: ICRA, Shanghai, China
Bradski G (2000) The OpenCV library. Dr Dobb’s journal of software tools
Collet Romea A, Martinez Torres M, Srinivasa S (2011) The MOPED framework: object recognition and pose estimation for manipulation. Int J Robot Res 30(10):1284–1306
Duncan K, Sarkar S, Alqasemi R, Dubey R (2013) Multi-scale superquadric fitting for efficient shape and pose recovery of unknown objects. In: IEEE international conference on robotics and automation (ICRA)
Ferrucci D, Lally A (2004) UIMA: an architectural approach to unstructured information processing in the corporate research environment. Nat Lang Eng 10(3–4):327–348
Ferrucci D, Nyberg E, Allan J, Barker K, Brown E, Chu-Carroll J, Ciccolo A, Duboue P, Fan J, Gondek D, Hovy E, Katz B, Lally A, McCord M, Morarescu P, Murdock B, Porter B, Prager J, Strzalkowski T, Welty C, Zadrozny W (2009) Towards the open advancement of question answering systems. Technical report RC24789, IBM Research Report
Ferrucci D, Brown E, Chu-Carroll J, Fan J, Gondek D, Kalyanpur AA, Lally A, Murdock JW, Nyberg E, Prager J, Schlaefer N, Welty C (2010) Building Watson: an overview of the DeepQA project. AI Mag 31(3):59–79. http://www.aaai.org/ojs/index.php/aimagazine/article/view/2303
Gould S, Russakovsky O, Goodfellow I, Baumstarck P, Ng AY, Koller D (2010) The stair vision library (v2.4). http://ai.stanford.edu/sgould/svl
Hinterstoisser S, Lepetit V, Ilic S, Holzer S, Bradski G, Konolige K, , Navab N (2012) Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. In: Asian conference on computer vision
Kragic D, Vincze M (2009) Vision for robotics. Found Trends Robot 1(1):1–78. doi:10.1561/2300000001
Lai K, Bo L, Ren X, Fox D (2011) Sparse distance learning for object recognition combining RGB and depth information. In: Proceedings of international conference on robotics and automation (ICRA)
Marton ZC, Pangercic D, Blodow N, Beetz M (2011) Combined 2D–3D categorization and classification for multimodal perception systems. Int J Robot Res 30(11):1378–1402
Marton ZC, Seidel F, Balint-Benczedi F, Beetz M (2012a) Ensembles of strong learners for multi-cue classification. Pattern recognition letters (PRL), Special issue on scene understandings and behaviours analysis
Marton ZC, Seidel F, Beetz M (2012b) Towards modular spatio-temporal perception for task-adapting robots. In: Postgraduate conference on robotics and development of cognition (RobotDoC-PhD), a satellite event of the 22nd international conference on artificial neural networks (ICANN). Lausanne, Switzerland
Mörwald T, Prankl J, Richtsfeld A, Zillich M, Vincze M (2010) Blort- the blocks world robotic vision toolbox. In: “Best practice algorithms in 3D perception and modeling for mobile manipulation workshop”—CD (in conjunction with the IEEE ICRA 2010)
Muja M, Rusu RB, Bradski G, Lowe D (2011) REIN—A fast, robust, scalable REcognition INfrastructure. In: ICRA. Shanghai, China
Nyga D, Balint-Benczedi F, Beetz M (2014) PR2 Looking at things: ensemble learning for unstructured information processing with Markov logic networks. In: IEEE international conference on robotics and automation (ICRA). Hong Kong, China
Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42:145–175
Pangercic D, Tenorth M, Jain D, Beetz M (2010) Combining perception and knowledge processing for everyday manipulation. In: IEEE/RSJ international conference on intelligent robots and systems (IROS). Taipei, Taiwan, pp 1065–1071
Pangercic D, Mathe K, Marton ZC, Goron LC, Opris MS, Schuster M, Tenorth M, Jain D, Ruehr T, Beetz M (2011) A robot that shops for and stores groceries. AAAI video competition (AIVC 2011). http://youtu.be/x0Ybod_6ADA
Pangercic D, Tenorth M, Pitzer B, Beetz M (2012) Semantic object maps for robotic housework—representation, acquisition and use. In: 2012 IEEE/RSJ international conference on intelligent robots and systems (IROS). Vilamoura, Portugal
Richardson M, Domingos P (2006) Markov logic networks. Mach Learn 62(1–2):107–136. doi:10.1007/s10994-006-5833-1
Rühr T, Sturm J, Pangercic D, Cremers D, Beetz M (2012) A generalized framework for opening doors and drawers in kitchen environments. In: IEEE International conference on robotics and automation (ICRA). St. Paul, MN, USA
Rusu RB, Cousins S (2011) 3D is here: point cloud library (PCL). In: IEEE international conference on robotics and automation (ICRA). Shanghai, China, pp 1–4
Rusu RB, Meeussen W, Chitta S, Beetz M (2009) Laser-based perception for door and handle identification. In: Proceedings of the international conference on advanced robotics (ICAR), Munich, Best Paper Award
Rusu RB, Bradski G, Thibaux R, Hsu J (2010) Fast 3d recognition and pose using the viewpoint feature histogram. In: Proceedings of the 23rd IEEE/RSJ IROS. Taipei, Taiwan
Tenorth M, Beetz M (2013) Knowrob—a knowledge processing infrastructure for cognition-enabled robots. Int J Robot Res (IJRR) 32(5):566–590
Tenorth M, Klank U, Pangercic D, Beetz M (2011) Web-enabled robots—robots that use the web as an information resource. Robot Autom Mag 18(2):58–68
Tenorth M, Profanter S, Balint-Benczedi F, Beetz M (2013) Decomposing cad models of objects of daily use and reasoning about their functional parts. In: IEEE/RSJ International conference on intelligent robots and systems (IROS). Tokyo Big Sight, Japan
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Beetz, M. et al. (2015). RoboSherlock: Unstructured Information Processing Framework for Robotic Perception. In: Busoniu, L., Tamás, L. (eds) Handling Uncertainty and Networked Structure in Robot Control. Studies in Systems, Decision and Control, vol 42. Springer, Cham. https://doi.org/10.1007/978-3-319-26327-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-26327-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26325-0
Online ISBN: 978-3-319-26327-4
eBook Packages: EngineeringEngineering (R0)