Direkt zum Inhalt

Owner only: item control page
Schmid, Andreas ; Fischer, Thomas ; Weichart, Alexander ; Hartmann, Alexander ; Wimmer, Raphael

ScreenshotMatcher: Taking Smartphone Photos to Capture Screenshots

Schmid, Andreas , Fischer, Thomas, Weichart, Alexander, Hartmann, Alexander and Wimmer, Raphael (2021) ScreenshotMatcher: Taking Smartphone Photos to Capture Screenshots. In: Mensch und Computer 2021 (MuC ’21), September 5–8, 2021, Ingolstadt, Germany.

Date of publication of this fulltext: 11 Aug 2021 04:15
Conference or workshop item
DOI to cite this document: 10.5283/epub.47814


Abstract

Taking screenshots is a common way of capturing screen content to share it with others or save it for later. Even though all major desktop operating systems come with a screenshot function, a lot of people also use smartphone cameras to photograph screen contents instead. While users see this method as faster and more convenient, image quality is significantly lower. With ScreenshotMatcher, we ...

Taking screenshots is a common way of capturing screen content to share it with others or save it for later. Even though all major desktop operating systems come with a screenshot function, a lot of people also use smartphone cameras to photograph screen contents instead. While users see this method as faster and more convenient, image quality is significantly lower. With ScreenshotMatcher, we present a system that allows for capturing a high-fidelity screenshot by taking a smartphone photo of (part of) the screen. A smartphone application sends a photo of the screen region of interest to a program running on the PC which retrieves the matching screen region and sends it back to the smartphone. Comparing four feature matching algorithms and multiple parameters, we identified a combination of ORB keypoint detection (feature limit 2000) and a brute force feature matcher using Hamming distance as the best solution for this task (success rate: 85%, processing time: 90 ms). This raw performance results in a real-world success rate of 47% and a mean response time per screenshot of 878 ms as measured in a remote user study (N=19). Released as open-source code, ScreenshotMatcher may be used as a basis for applications and research prototypes that bridge the gap between PC and smartphone.



Involved Institutions


Details

Item typeConference or workshop item (Paper)
ISBN978-1-4503-8645-6
Journal or Publication TitleProceedings of the Mensch und Computer 2021
Title of Book:MuC '21: Proceedings of Mensch und Computer 2021
Publisher:Association for Computing Machinery
Open Access Type:Assoc. of Comp. Machinery (ACM)
Place of Publication:New York, United States
Page Range:pp. 44-48
Date5 September 2021
InstitutionsLanguages and Literatures > Institut für Information und Medien, Sprache und Kultur (I:IMSK) > Lehrstuhl für Medieninformatik (Prof. Dr. Christian Wolff)
Informatics and Data Science > Department Human-Centered Computing > Lehrstuhl für Medieninformatik (Prof. Dr. Christian Wolff)

Languages and Literatures > Institut für Information und Medien, Sprache und Kultur (I:IMSK) > Lehrstuhl für Medieninformatik (Prof. Dr. Christian Wolff) > Physical-Digital Affordances (Dr. Raphael Wimmer)
Informatics and Data Science > Department Human-Centered Computing > Lehrstuhl für Medieninformatik (Prof. Dr. Christian Wolff) > Physical-Digital Affordances (Dr. Raphael Wimmer)
Informatics and Data Science > Department Human-Centered Computing > Physical-Digital Affordances (Dr. Raphael Wimmer)
Identification Number
ValueType
10.1145/3473856.3474014DOI
Related URLs
URLURL Type
https://hci.ur.de/projects/screenshotmatcherProject
Keywordsmobile, computer vision, cross device interaction
Dewey Decimal Classification000 Computer science, information & general works > 004 Computer science
StatusPublished
RefereedYes, this version has been refereed
Created at the University of RegensburgYes
URN of the UB Regensburgurn:nbn:de:bvb:355-epub-478142
Item ID47814

Export bibliographical data

Owner only: item control page

nach oben