Emulating Web Browser in a Dedicated Intermediary Box

Abstract

Anonymity and privacy of a client device that fetches a content from a web server are improved by using an intermediate device located along the communication path between the client device and the web server. The primary or exclusive function of the intermediate device may be to serve as an intermediate device, and may be implemented as a stand-alone dedicated client device located at a residential premises, or may be integrated with another device, such as a router or a sensor unit, and may communicate using wired communication (such as LAN) or wireless communication (such as WLAN). The intermediate device may modify a content request from the client device in order to avoid identification or blocking by a web server that uses web tracking, such as fingerprinting. The modification may use a web browser, such as a headless browser, for emulating a different device or user.

1 . A method for use with a web server that stores a content identified by a Uniform Resource Locator (URL) the method comprising:
 receiving, by a first client device from a first server over the Internet, a first request that includes the URL;   extracting, by the first client device, the URL from the first request;   forming, by the first client device, a second request that includes the URL, or modifying the first request to obtain the second request;   sending, by the first client device to the second client device over the Internet, the second request;   receiving, by the second client device from the first client device over the Internet, the second request; and   sending, by the second client device to the web server over the Internet, the received second request,   wherein the sending the received second request to the web server uses an IP address of the second client device, so that the IP address of the first client device is unknown to the web server, and   wherein, except for using the IP address of the second client device as a source address, the second client device transparently passes messages between the first client device and the web server.   
     
     
         2 . The method according to  claim 1 , further comprising:
 receiving, by the second client device from the web server over the Internet, the content, in response to the sending of the second request;   sending, by the second client device to the first client device over the Internet, the received content;   receiving, by the first client device from the second client device over the Internet, the received content; and   sending, by the first client device to the first server over the Internet, the received content.   
     
     
         3 . The method according to  claim 1 , wherein the web server uses or executes a web tracker for blocking requests that satisfy a criterion, and wherein the second request is formed so that it does not satisfy the criterion, so that the second request is not blocked by the web tracker in the web server. 
     
     
         4 . The method according to  claim 3 , for use with a group of entities that comprise devices, users, user activities, software applications, and user agents, wherein the web tracker is configured to identify an entity associated with the first client device, and wherein the criterion is satisfied if the identified entity associated with the first client device is included in the group. 
     
     
         5 . The method according to  claim 4 , wherein the web tracker is configured to identify or re-identify an entity using fingerprinting. 
     
     
         6 . The method according to  claim 1 , further for overcoming a blocking of requests by the web server, or for improving anonymity or privacy of the first client device, a user of the first client device, the second client device, a user of the second client device, or any combination thereof. 
     
     
         7 . The method according to  claim 1 , wherein the first client device comprises a non-transitory computer-readable medium that contains computer instructions that, when executed by a computer processor, cause the processor to perform at least part of the steps of  claim 1 . 
     
     
         8 . The method according to  claim 1 , wherein at least part of steps of  claim 1  are included in a Software Development Kit (SDK) that is provided as a non-transitory computer-readable medium containing computer instructions, and wherein the method further comprising installing the SDK. 
     
     
         9 . The method according to  claim 1 , wherein the content comprises, or consists of, a HyperText Markup Language (HTML) object, a web-page, a web-site, or any combination thereof, that includes, consists of, or comprises, a part or whole of a program or data file, text data, audio data, voice data, multimedia data, video data, an image, music data, or any combination thereof. 
     
     
         10 . The method according to  claim 1 , further comprising receiving, by the first client device over the Internet, the IP address of the second client device, and wherein the sending of the second request by the first client device to the second client device is in response to the receiving of the IP address of the second client device. 
     
     
         11 . The method according to  claim 10 , further comprising sending, by the first client device over the Internet, a request for the IP address of the second client device, and wherein the receiving of the IP address of the second client device is in response to the sending of the request for the IP address. 
     
     
         12 . The method according to  claim 10 , wherein the receiving of the IP address of the second client device is from the first client device or from the first server or a second server that is different from the first server. 
     
     
         13 . The method according to  claim 10 , for use with a group of client devices that includes the second client device, the method further comprising selecting the second client device from the group of client devices. 
     
     
         14 . The method according to  claim 13 , wherein IP addresses of the client devices in the group are stored in the first client device, and wherein the selecting is by the first client device. 
     
     
         15 . The method according to  claim 14 , further comprising receiving and storing of the IP addresses of the client devices in the group. 
     
     
         16 . The method according to  claim 15 , wherein the receiving of the IP addresses of the client devices in the group is from the first client device or from a server device. 
     
     
         17 . The method according to  claim 13 , wherein IP addresses of the client devices in the group are stored in a server device, and wherein the selecting is by the server device. 
     
     
         18 . The method according to  claim 13 , wherein the selecting is in response to the URL, a domain in the first request, a web-page in the first request, a web-site in the first request, or any combination thereof. 
     
     
         19 . The method according to  claim 13 , wherein the selecting is based on a time of the selecting or is time-based. 
     
     
         20 . The method according to  claim 13 , wherein the selecting is based on a calendar time. 
     
     
         21 . The method according to  claim 20 , wherein the selecting is based on a calendar month, a week, a day of the week, an hour of a day, a minute in an hour, or any combination thereof. 
     
     
         22 . The method according to  claim 13 , wherein the selecting uses, or is based on, an action or an event that is external to, and sensed by, the first client device. 
     
     
         23 . The method according to  claim 13 , wherein the selecting uses, or is based on, load balancing, or wherein the selecting uses, or is based on, random, quazi-random, or deterministic selection. 
     
     
         24 . The method according to  claim 23 , wherein the selecting uses, or is based on, random selecting that uses one or more random numbers generated by a random number generator. 
     
     
         25 . The method according to  claim 24 , wherein the random number generator is hardware-based that uses, or is based on, thermal noise, shot noise, nuclear decaying radiation, photoelectric effect, or quantum phenomena. 
     
     
         26 . The method according to  claim 24 , wherein the random number generator is software-based that uses, or is based on, executing an algorithm for generating pseudo-random numbers. 
     
     
         27 . The method according to  claim 13 , wherein the IP addresses are arranged in a sequence in the group, and wherein the selecting is based on, or uses, sequential selection, cyclic selection, Last-In-First-Out (LIFO), First-In-First-Out (FIFO) scheme, or any combination thereof. 
     
     
         28 . The method according to  claim 13 , for use with a first attribute type, and wherein each of the client devices in the group is associated with a first value relating to the first attribute type that comprises a numeric value or an identifier of a feature, an attribute, a characteristic, or a property of the first attribute type, wherein the selecting comprises selecting based on the first value associated with the selected client device. 
     
     
         29 . The method according to  claim 28 , wherein the first attribute type comprises a geographical location, and wherein each of the first values comprises a name or an identifier of a continent, a country, a region, a city, a street, a ZIP code, or a timezone. 
     
     
         30 . The method according to  claim 29 , wherein the first value of each of the client devices in the group or each of the IP addresses is based on IP geolocation. 
     
     
         31 . The method according to  claim 30 , wherein the geolocation is based on W3C Geolocation API. 
     
     
         32 . The method according to  claim 30 , for use with a database that associates IP addresses to geographical locations. 
     
     
         33 . The method according to  claim 32 , wherein the database is stored in a first server. 
     
     
         34 . The method according to  claim 33 , further comprising receiving and storing, by the first server, the database. 
     
     
         35 . The method according to  claim 33 , further comprising estimating or associating the first value to each of the client devices in the group by the database. 
     
     
         36 . The method according to  claim 28 , wherein the first attribute type comprises Internet Service Provider (ISP) or Autonomous System Number (ASN), wherein each of the first values comprises respectively a name or an identifier of the ISP or the ASN number. 
     
     
         37 . The method according to  claim 28 , wherein the first attribute type corresponds to a hardware of software of client devices. 
     
     
         38 . The method according to  claim 37 , wherein the first attribute type comprises the hardware of the client devices. 
     
     
         39 . The method according to  claim 38 , wherein the first values comprise stationary or portable values, respectively based on the client device being stationary or portable. 
     
     
         40 . The method according to  claim 37 , wherein the first attribute type comprises a software application installed, used, or operated, in client devices. 
     
     
         41 . The method according to  claim 40 , wherein the first values comprise the type, make, model, or version of the software, or wherein the software application comprises an operating system. 
     
     
         42 . The method according to  claim 28 , wherein the first attribute type corresponds to a communication property, feature of a communication link of client devices. 
     
     
         43 . The method according to  claim 42 , wherein the communication link corresponds to the respective connection to the Internet of client devices, or wherein the communication link corresponds to a communication link of a client device with the first client device or with the web server. 
     
     
         44 . The method according to  claim 42 , wherein the first attribute type corresponds to a bandwidth (BW) or Round-Trip delay Time (RTT) of the communication link, and the first value is the respective estimation or measurement of the BW or RTT. 
     
     
         45 . The method according to  claim 44 , further comprising estimating or measuring, by the first server or by a client device, the BW or RTT of the communication link. 
     
     
         46 . The method according to  claim 42 , wherein the first attribute type corresponds to a technology or scheme used by the client devices for connecting to the Internet. 
     
     
         47 . The method according to  claim 46 , wherein the first values comprise wired or wireless values, respectively based on the client device being connected to the Internet using a wired or wireless connection. 
     
     
         48 . The method according to  claim 1 , wherein the web server uses or executes a web tracker for blocking requests that satisfy a criterion, wherein the web tracker uses, or is based on, fingerprinting for identifying or re-identifying devices, users, user activities, software applications or instances, user agents, or any combination thereof, wherein the web tracker obtains a first fingerprint from the first client device, and for use with a second fingerprint that is different from the first fingerprint, the method further comprising modifying, by the first client device, from the first fingerprint to the second fingerprint, or wherein the second request is associated with, or is based on, the second fingerprint. 
     
     
         49 . The method according to  claim 48 , wherein the first fingerprint satisfies the criterion. 
     
     
         50 . The method according to  claim 49 , wherein the second fingerprint does not satisfy the criterion. 
     
     
         51 . The method according to  claim 48 , wherein the fingerprinting comprises passive fingerprinting, active fingerprinting, cookie-like fingerprinting, or any combination thereof. 
     
     
         52 . The method according to  claim 48 , wherein the modifying comprises injecting a script into a network payload that is directed to the client device, where the script is overriding an application programming interface call that is configured to collect fingerprinting information. 
     
     
         53 . The method according to  claim 48 , wherein the modifying comprises modifying tan intercepted network traffic by obfuscating a fingerprinting object that the client device is attempting to upload to the web browser fingerprinting service. 
     
     
         54 . The method according to  claim 48 , wherein the modifying comprises Open Systems Interconnection (OSI) model Layer-2 modifying, OSI Layer-3 modifying, OSI Application Layer modifying, OSI Session Layer modifying, or any combination thereof. 
     
     
         55 . The method according to  claim 48 , wherein the fingerprinting comprises, or is based on, device fingerprinting that comprises a systematic gathering of a set of information from a requesting client device for differentiating it for identifying, monitoring, or blocking a requesting client activity. 
     
     
         56 . The method according to  claim 55 , wherein the fingerprinting uniquely identifies the requesting client device. 
     
     
         57 . The method according to  claim 55 , wherein the device fingerprinting comprises, or is based on, Web-based fingerprinting. 
     
     
         58 . The method according to  claim 55 , wherein the Web-based fingerprinting comprises, or is based on, browser fingerprinting that comprises retrieving information by interacting with a web browser that is executed by the first or second client device that requests a content from the web server. 
     
     
         59 . The method according to  claim 58 , wherein the browser fingerprinting comprises, or is based on, a single browser fingerprinting. 
     
     
         60 . The method according to  claim 58 , wherein the browser fingerprinting comprises, or is based on, using javascript, Flash Plugin, Java Plugin, Java Virtual Machine (JVM), Cascading Style Sheets (CSS), or any combination thereof. 
     
     
         61 . The method according to  claim 58 , wherein the browser fingerprinting comprises, or is based on, cross-browser fingerprinting. 
     
     
         62 . The method according to  claim 58 , wherein the web browser supports HyperText Markup Language 5 (HTML5). 
     
     
         63 . The method according to  claim 62 , wherein the browser fingerprinting comprises, or is based on, canvas fingerprinting that comprises identifying and tracking requesting client devices that use HyperText Markup Language 5 (HTML5) canvas elements. 
     
     
         64 . The method according to  claim 48 , wherein the web tracker is based on, or comprises, fingerprinting that use at least one value that is associated with a feature, property, characteristic, or attribute of the first client device or of the first client device, and wherein the method further comprising modifying or changing the value to form a second fingerprint data that is different from the first fingerprint. 
     
     
         65 . The method according to  claim 64 , wherein the first fingerprint satisfies the criterion and wherein the second fingerprint does not satisfy the criterion. 
     
     
         66 . The method according to  claim 64 , wherein first fingerprinting comprises browser fingerprinting, canvas fingerprinting, canvas font fingerprinting, webRTC fingerprinting, audiocontext fingerprinting, or any combination thereof. 
     
     
         67 . The method according to  claim 64 , wherein the value is associated with type, version, characteristics, or configuration of a software application installed or used in the first or second client device or in the first or second client device. 
     
     
         68 . The method according to  claim 67 , wherein the software application comprises a web browser, a graphic driver, or an operating system. 
     
     
         69 . The method according to  claim 67 , wherein the value comprises a vendor name or identifier of the software application. 
     
     
         70 . The method according to  claim 67 , wherein the value comprises a list of plugins activated in a web-browser, whether cookies are enabled or not in a web-browser, Time-zone used by a web-browser, or any combination thereof. 
     
     
         71 . The method according to  claim 64 , wherein the value is associated with user configuration of hardware or software in the first client device or in the first client device. 
     
     
         72 . The method according to  claim 71 , wherein the value is associated with a number of software applications installed, a language used, a time zone, an installation of an ad-blocker, fonts used, cached objects, or the screen configuration. 
     
     
         73 . The method according to  claim 71 , wherein the value is associated with an HTTP cache, browsing history, Local Shared Objects (Flash Cookies), Web Storage (HTML5 Cookies), or any combination thereof. 
     
     
         74 . The method according to  claim 64 , wherein the value is associated with a hardware component of the first client device or of the first client device, a hardware arrangement of the first client device or of the first client device, or one or more components that are connected to the first client device or to the first client device. 
     
     
         75 . The method according to  claim 74 , wherein the value is associated with an existence of, or with a property, configuration, characteristic, or attribute of, a graphic card, a memory, an accelerometer, a network interface, a touch-screen, a screen, a battery, an audio codec, a video codec, or any combination thereof. 
     
     
         76 . The method according to  claim 75 , wherein the value is associated with a resolution of the screen or of the touch-screen or with a clock skew. 
     
     
         77 . The method according to  claim 75 , wherein the value comprises a vendor name or identifier of the hardware component. 
     
     
         78 . The method according to  claim 75 , wherein the value comprises a number of processors, an amount of free memory space, an amount of maximum available memory, an amount of total memory, or any combination thereof. 
     
     
         79 . The method according to  claim 64 , wherein the value is associated with a value in a HTTP header field. 
     
     
         80 . The method according to  claim 79 , wherein the header field comprises User Agent, Accept Header, Accept-Charset Header, Accept-Encoding Header, Do-not-track Header, or any combination thereof. 
     
     
         81 . The method according to  claim 64 , wherein the fingerprinting comprises, or is based on, browser fingerprinting that comprises retrieving information from a web browser that is executed by a requesting client device that fetches content from the web server. 
     
     
         82 . The method according to  claim 81 , wherein the information retrieved is hardware, operating-system, or web-browser related. 
     
     
         83 . The method according to  claim 81 , wherein the modifying from the first fingerprint to a second fingerprint comprises modifying or forming a User-agent header, an Accept header, a Connection header, an Encoding header, a Language header, a list of plugins, a computing platform, a cookies preferences (allowed or not), a ‘Do Not Track’ preferences (yes, no or not communicated), a timezone, a screen resolution and its color depth, a use of local storage, a use of session storage, a picture rendered with the HTML Canvas element, a picture rendered with WebGL, a presence of AdBlock, a list of fonts, or any combination thereof. 
     
     
         84 . The method according to  claim 81 , wherein the modifying comprises using or executing a web browser by the first or second client device. 
     
     
         85 . The method according to  claim 81 , wherein the information is retrieved using a JavaScript script executed by the web browser or by using an Application Programming Interface (API). 
     
     
         86 . The method according to  claim 85 , wherein the retrieved information comprises List of Plugins, Cookies Enabled, Use of Local or Session Storage, Timezone, Screen Resolution and Color Depth, List of Fonts, Platform, ‘Do-Not-Track’ status, Canvas, WebGL Vendor, WebGL Renderer, Use of Ad Blocker, or any combination thereof. 
     
     
         87 . The method according to  claim 64 , wherein the web server or the first client device supports HyperText Markup Language 5 (HTML 5). 
     
     
         88 . The method according to  claim 87 , wherein the fingerprinting comprises, or is based on, canvas fingerprinting that comprises identifying and tracking visitors using HTML5 canvas element. 
     
     
         89 . The method according to  claim 88 , wherein the modifying from the first fingerprint to a second fingerprint comprises modifying or forming UserAgent; Language; Color Depth; Screen Resolution; Timezone; having session storage or not; having local storage or not; having indexed DB, having IE specific ‘AddBehavior’, having open DB; CPU class; Platform; DoNotTrack or not; full list of installed fonts, implemented with Flash; a list of installed fonts, detected with JS/CSS (side-channel technique); WebGL fingerprinting; Plugins (IE included); AdBlock installed or not; having the user tampered with its languages; having the user tampered with its screen resolution; having the user tampered with its OS; having the user tampered with its browser; touch screen detection and capabilities; Pixel Ratio; System's total number of logical processors available to the user agent; or any combination thereof. 
     
     
         90 . The method according to  claim 88 , wherein the modifying comprises using or executing a web browser by the first or second client device. 
     
     
         91 . The method according to  claim 1 , wherein the second request is formed or generated by modifying a header field in the first request. 
     
     
         92 . The method according to  claim 1 , wherein the first request comprises, or consists of, a Hypertext Transfer Protocol (HTTP) request, and the HTTP is based on, comprises, or consists of, HTTP/1.1, HTTPS, HTTP/2, HTTP/3, or any combination thereof. 
     
     
         93 . The method according to  claim 92 , wherein the HTTP request comprises, or consists of, Hypertext Transfer Protocol Secure (HTTPS) request. 
     
     
         94 . The method according to  claim 92 , wherein the second request is formed or generated by modifying a header field in the HTTP request. 
     
     
         95 . The method according to  claim 92 , wherein the second request comprises, or consists of, a Hypertext Transfer Protocol (HTTP) request. 
     
     
         96 . The method according to  claim 91 , wherein the second request comprises, or consists of, a Hypertext Transfer Protocol (HTTP) request, and wherein at least one header field is modified from the first to second HTTP requests. 
     
     
         97 . The method according to  claim 96 , wherein the modified header field comprises User-Agent field, Accept field, Content Encoding field, Content Language field.