Background of the project

In order to meet the needs of monitoring user screenshots and displaying the floating feedback portal, we conducted a simple survey on the user screenshot function on the Android side. Since the Android system does not provide APIs related to screen capture notifications, we need to make use of the relevant features that the system can provide to implement them.

Through study, I saw that there are probably three solutions on the Internet:

1. Use FileObserver to monitor resource changes in a directory
  2. Use ContentObserver to monitor changes in image resources
  3. Monitor shortcut keys for screenshots (due to the diversity of Android systems customized by manufacturers, plus the difference in shortcut keys and third-party applications, it is basically unreliable to monitor shortcut keys for screenshots, and can be ignored directly)

My implementation here is solved through the second solution. I have tested a variety of models and used them online, and there is no problem for now.

Principle introduction

As we all know, the Android system has a media database. Whether we take a picture with a camera or use a system screenshot, the system will add the detailed information of this picture to the media database and issue a content change notification, so we can Use the Content Observer (ContentObserver) to monitor the changes in the media database. When the database changes, get the last inserted picture data. If the picture meets our specific rules, it will be considered as a screenshot.

So what do we need to do to make sure it is a screenshot?

1. Monitor the resource URI of the screenshot (MediaStore.Images.Media.EXTERNAL_CONTENT_URI)
2. Because we want to read the content of the picture, we need to read the permission of the SD card, android.permission.READ_EXTERNAL_STORAGE and apply dynamically.
  3. After obtaining the picture information, judge whether the picture complies with the screenshot rules.

screenshot rules

Most of the rules on the Internet:

1. Time judgment, the generation time of the picture is after the start of monitoring, and within 10 seconds of the current time: the picture generated after the start of monitoring is meaningful, and within 10 seconds means that it was just generated.
  2. Judging the path, the image path conforms to include specific keywords: This is the key point, the save path of the screenshot image usually contains screenshot strings such as "screenshot".

Implement the main steps

Register Image Listener

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
fun registerContentObserver() {
        if (contentObserver == null) {
            contentObserver =
                ScreenShotApplication.applicationContext.contentResolver.registerObserver(
                    MediaStore.Images.Media.EXTERNAL_CONTENT_URI
                ) {
                    _dataChanged.value = true
                }
        }
    }


/**
 * Use ContentResolver to monitor changes in photo data
 */
fun ContentResolver.registerObserver(
        uri: Uri,
        observer: (selfChange: Boolean) -> Unit
): ContentObserver {
    val contentObserver = object : ContentObserver(Handler()) {
        override fun onChange(selfChange: Boolean) {
            observer(selfChange)
        }
    }
    registerContentObserver(uri, true, contentObserver)
    return contentObserver
}

Get screenshot image

After adapting to Android 11, the SQL for querying pictures has changed, so some changes need to be made to the query method, see the code below for details.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
/*
 *  Get screenshot image
 */
 fun getScreentShotImage(bucketId: String? = null) {
     Thread {
         try {
             var data: ScreentShotInfo? = null
             if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
                 //11 
                 data = queryImagesP(bucketId)
             } else {
                 data = queryImages(bucketId)
             }
             val imagePath = data.path?.toLowerCase()
             screenShoot.forEach {
                 if (imagePath?.contains(it)!! && (System.currentTimeMillis() / 1000 - data.addTime < 2)) {
                     _screentShotInfoData.postValue(data)
                     return@forEach
                 }
             }
         } catch (e: Exception) {
         }
     }.start()
 }

 /**
  * Only get ordinary pictures, not Gif
  */
 fun queryImages(bucketId: String?): ScreentShotInfo {
     val screentShotInfo = ScreentShotInfo()

     val uri = MediaStore.Files.getContentUri("external")
     val sortOrder = MediaStore.Images.Media._ID + " DESC limit 1 "
     var selection = (MediaStore.Files.FileColumns.MEDIA_TYPE + "="
             + MediaStore.Files.FileColumns.MEDIA_TYPE_IMAGE) +
             " AND " + MediaStore.Images.Media.MIME_TYPE + "=?" +
             " or " + MediaStore.Images.Media.MIME_TYPE + "=?"
     try {
         val data = ScreenShotApplication.applicationContext.contentResolver.query(
             uri,
             ScreenShotProjection,
             selection,
             imageType,
             sortOrder
         )

         if (data == null) {
             return screentShotInfo
         }

         if (data.moveToFirst()) {
             val imageId: String =
                 data.getString(data.getColumnIndexOrThrow(ScreenShotProjection[0]))
             val imagePath: String =
                 data.getString(data.getColumnIndexOrThrow(ScreenShotProjection[1]))
             val imageSize: Long =
                 data.getLong(data.getColumnIndexOrThrow(ScreenShotProjection[2]))
             val imageWidth: Int =
                 data.getInt(data.getColumnIndexOrThrow(ScreenShotProjection[3]))
             val imageHeight: Int =
                 data.getInt(data.getColumnIndexOrThrow(ScreenShotProjection[4]))
             val imageMimeType: String =
                 data.getString(data.getColumnIndexOrThrow(ScreenShotProjection[5]))
             val imageAddTime: Long =
                 data.getLong(data.getColumnIndexOrThrow(ScreenShotProjection[6]))
             screentShotInfo.path = imagePath
             screentShotInfo.addTime = imageAddTime
         }

     } catch (e: Exception) {
         e.printStackTrace()
     }

     return screentShotInfo
 }

	/**
   * Only get ordinary pictures, not Gif (in Android11 machines)
   * After the targetSdkVersion is adapted to 30, the Sql for querying pictures has changed
   */
 @RequiresApi(Build.VERSION_CODES.O)
 @WorkerThread
 fun queryImagesP(bucketId: String?): ScreentShotInfo {
     val screentShotInfo = ScreentShotInfo()
     val uri = MediaStore.Files.getContentUri("external")
     val sortOrder = MediaStore.Files.FileColumns._ID + " DESC"
     var selection = (MediaStore.Files.FileColumns.MEDIA_TYPE + "="
             + MediaStore.Files.FileColumns.MEDIA_TYPE_IMAGE) +
             " AND " + MediaStore.Images.Media.MIME_TYPE + "=?" +
             " or " + MediaStore.Images.Media.MIME_TYPE + "=?"

     val bundle = createSqlQueryBundle(selection, imageType, sortOrder, 1)

     try {
         val data = ScreenShotApplication.applicationContext.contentResolver.query(
             uri,
             ScreenShotProjection,
             bundle,
             null
         )

         if (data == null) {
             return screentShotInfo
         }

         if (data.moveToFirst()) {
             val imageId: String =
                 data.getString(data.getColumnIndexOrThrow(ScreenShotProjection[0]))
             val imagePath: String =
                 data.getString(data.getColumnIndexOrThrow(ScreenShotProjection[1]))
             val imageSize: Long =
                 data.getLong(data.getColumnIndexOrThrow(ScreenShotProjection[2]))
             val imageWidth: Int =
                 data.getInt(data.getColumnIndexOrThrow(ScreenShotProjection[3]))
             val imageHeight: Int =
                 data.getInt(data.getColumnIndexOrThrow(ScreenShotProjection[4]))
             val imageMimeType: String =
                 data.getString(data.getColumnIndexOrThrow(ScreenShotProjection[5]))
             val imageAddTime: Long =
                 data.getLong(data.getColumnIndexOrThrow(ScreenShotProjection[6]))
             screentShotInfo.path = imagePath
             screentShotInfo.addTime = imageAddTime
         }

     } catch (e: Exception) {
         e.printStackTrace()
     }
     return screentShotInfo
 }

 /*
 * Create the bundle object required by Android11
 * */
 fun createSqlQueryBundle(
     selection: String,
     selectionArgs: Array<String>,
     sortOrder: String?, limitCount: Int = 0, offset: Int = 0
 ): Bundle? {
     if (selection == null && selectionArgs == null && sortOrder == null) {
         return null
     }
     val queryArgs = Bundle()
     if (selection != null) {
         queryArgs.putString(ContentResolver.QUERY_ARG_SQL_SELECTION, selection)
     }
     if (selectionArgs != null) {
         queryArgs.putStringArray(ContentResolver.QUERY_ARG_SQL_SELECTION_ARGS, selectionArgs)
     }
     if (sortOrder != null) {
         queryArgs.putString(ContentResolver.QUERY_ARG_SQL_SORT_ORDER, sortOrder)
     }
     queryArgs.putString(ContentResolver.QUERY_ARG_SQL_LIMIT, "$limitCount offset $offset")
     return queryArgs
 }

write at the end

So far, our needs have been met. Some inexplicable null pointer problems were found in the crash monitoring background, so some mandatory null judgment and try catch processing were added to the code. There will be time to optimize in the follow-up, and I hope everyone can provide valuable opinions.