Showing revision #27558534 of page emoticons-guides

emoticons-guides

     ░ ▓ ▒                      ░          ░                          ▒ ▓ ░     
     ░ ▓                         ░  ███▄  ░                             ▓ ░     
     ░                             ██∙███                                 ░     
          █▄ ▄█│  ▄▄█████│       ░ ██  ██ ░   │███   │█████▄▄  ▀█   ▄           
     ████│██▄██│ ██═════▀█│ █████│  ▀█▀█   ░│██  │█ │█▀═════██ │██ │█ │███▄     
     █│   █│█ █│██       ██│  █│   █▌   ▐█ │██     │██       ██│█│█│█│█▀ ▐▀     
     ██▌  █│  █│██│      ██│  █│    █████  │██  │█ │██      │██│█│█│█│█▄        
     █│   █│  █│██│      ██│  ▐│     ███▀   │████  │██      │██│█ │██ │██▄      
     ▀███│▀   █│ ██     ██│          ███            │██     ██ │█  │██  │█      
        ▀         █████▀▀  ▌         ██            ▐  ▀▀█████       ▀ ▀██▀      
     ░             ▄▄    ▄▀           ▀█            ▀▄    ▄▄              ░     
     ░ ▓            ▀▀▀█▀              ▀              ▀█▀▀▀             ▓ ░     
     ░ ▓ ▒              ▀▀▄▄▄▄▄  . EMOTICONS .  ▄▄▄▄▄▀▀               ▒ ▓ ░     
     ░ ▓ ▒                   ▀▀▀▓███▓▄▄█▄▄▓███▓▀▀▀                    ▒ ▓ ░     
     ░ ▓ ▒                         ▀▀▀▀█▀▀▀▀                          ▒ ▓ ░     
     ░ ▓ ▒        :                                          :        ▒ ▓ ░     
     ░ ▓ ▒       :~~~~~~~~~~~~. PROUDLY PRESENTS .~~~~~~~~~~~~:       ▒ ▓ ░     
     ░ ▓ ▒        :                                          :        ▒ ▓ ░     
     ░ ▓ ▒                      emoticons-guides                      ▒ ▓ ░     
       ▓ ▒                 These are some guides that                 ▒ ▓       
         ▒                are related to computer and                 ▒         
                                   interests.                                   
                                                                                
                             Take a look around! :)                             
                  :                                          :                  
                 :~~~~~~~~~~~~~~~~~. TAGS .~~~~~~~~~~~~~~~~~~~:                 
                  :                                          :                  
                                   Extension                                    
                                     Piracy                                     
                                    Privacy                                     
                                       RE                                       
                                 Steganography                                  
                                      Util                                      
     ░            :                                          :            ░     
     ░ ▓         :~~~~~~~~~~~~~~~. WRITTEN BY .~~~~~~~~~~~~~~~:         ▓ ░      
     ░ ▓ ▒        :                                          :        ▒ ▓ ░     
     ░ ▓ ▒                        /u/emoticons                        ▒ ▓ ░     
     ░ ▓ ▒        :                                          :        ▒ ▓ ░     
     ░ ▓ ▒       :~~~~~~~~~~~~~~~. DISCLAIMER .~~~~~~~~~~~~~~~:       ▒ ▓ ░      
     ░ ▓ ▒        :                                          :        ▒ ▓ ░     
     ░ ▓ ▒         emoticons-guides is very basic information         ▒ ▓ ░     
     ░ ▓ ▒                 and maybe not all accurate                 ▒ ▓ ░     
  1. TLS Record Protocol

    1. The Record

    2. Handshake TLS1.2

    3. Examining TLS ClientHello [Privacy] (TODO)

    4. Additional links

    5. Information missing here

  2. HTTP/1.1 Webclient [Util]

    1. Common headers

    2. Common methods

    3. Sample requests

    4. Additional links

    5. Information missing here

  3. HTTP/1.1 Webserver [Util]

    1. Common headers

    2. Common statuses

    3. Sample responses

    4. Additional links

    5. Information missing here

  4. HTTP proxy service [Piracy]

    1. /etc/hosts

    2. Sample proxy

    3. Extension

  5. Using openssl [Util]

    1. Generate an RSA private key

    2. Generate an X509 root certificate

    3. Generate a X509v3 root certificate

    4. X509v3 Extensions

    5. Generate a X509 certificate chain

    6. Additional links

    7. Information missing here

    8. Related

  6. HTTPS proxy service [Piracy]

    1. Sample proxy

  7. HTTP/2 [Util] [Extension] (TODO)

  8. HTTP/3 [Util] [Extension] (TODO)

  9. WebSockets (TODO)

    1. Insecure WebSockets (TODO)

    2. Secure WebSockets (TODO)

    3. Additional links

  10. Examining DirectShow WindowsAPI [Privacy] (TODO)

  11. Examining the ZIP File format [Steganography] [Privacy] (TODO)

    1. Additional links

  12. Examining the PNG File format [Steganography] [Privacy]

    1. Steganography

    2. Fingerprinting

    3. Additional links

    4. Information missing here

  13. Changing File times [Privacy] (TODO)

    1. Windows (TODO)

    2. Linux

  14. DEFLATE [Util] (TODO)

    1. Additional links

  15. Make a Firefox Addon [Util]

    1. Content Scripts

    2. Intercept HTTP

    3. Install Temporary Addon

    4. Additional links

  16. Crack your First Software [RE] (TODO)

  17. Cracking Java Software [RE] (TODO)

  18. Cracking .NET Software [RE] (TODO)

  19. Making your First keygen [RE] (TODO)

  20. Cracking DRM [RE] (TODO)

TLS Record Protocol

it is used to verify the identity of servers with asymmetric key and to make a symmetric key exchange from master secrets. The master secret is split up into keys, MAC keys and if require IV keys for the client and server both. (example: using diffie hellman from client and server shared pre master secrets) bytes are big endian. also can do Alert Protocol and Change Cipher Spec Protocol. As of this day, TLS 1.3 is the most up to date of this protocol. TLS 1.3 simplifies handshake by a lot. TLS comes from SSL, but TLS is more better with more update encryption.

The Record

The record just a wrapper for TLSCompressed (same format as TLSPlaintext) structure, or a TLSCiphertext. the first byte shows the content type. The next byte show the major protocol version. the next byte show the minor protocol version. The next two show the length. Then follows the data in the record. this data is compressed using the algorithm from the handshake, but on starting handshake, there is no compression. Encryption is done after compression, depending on cipher spec. MAC is added to compressed records. MAC algorithm depend on cipher spec. MAC will be calculated before encryption. A record can have more than one message of the same content type or the partial content of single message.

Handshake TLS1.2

A handshake is done from a client first. and a handshake done on top of the record.

  1. ClientHello and ServerHello

Client and server decide on the TLS version, cipher suite and compression used. client and server share random values. a session identify assigns to the client if not connection before. this is always a random session identity. If a client shows a previous session identifier to the server, it can continue a session because the cryptography can be expensive, a session will make sure the key exchanges only done once.

  1. Server Certificate (optional)

server will send intermediate and its own certificate to verify identity. Intermediate can be zero or more. Can send the root certificate as well, but is not required. Server certificate will contain a server public key

  1. Server ServerKeyExchange (optional)

server sending a public key used to make the pre master secret. Only sent if server has no certificate or server's certificate is used for only signing

  1. Server CertificateRequest (optional)

If the server need to confirm client identity.

  1. Server ServerHelloDone

This tells client that server hello and server part of key exchange is finish

  1. Client Certificate (optional)

Client will send certificate to verify identity. Only if request by Server CertificateRequest. if client has no certificate, message sent with zero length in data. Certificate or certificate chain sent the same as if a server sending the chain. client certificate contains client public key

  1. Client ClientKeyExchange

for rsa client sends a pre master secret, encrypted in server certificate public key.

  1. Client CertificateVerify (optional)

If a client send a certificate, it is verified by the digital signature. Indicating the client does has the private key for its certificate. Only send if the certificate is not have fixed Diffie Hellman. a hash of all prior handshake message is sent

  1. Client ChangeCipherSpec

uses a different content type in the record. notify server that next messages will compress and encrypt with the chosen cipher suite and compression

  1. Client Finished

message encrypted with algorithms in the cipher spec. contains HMAC number generated from MD5 and SHA1 from master secret, "client finished" string, and hash of all prior handshake message

  1. Server ChangeCipherSpec

Using a different content type in the record. notify client that next messages will compress and encrypt with the chosen cipher suite and compression

  1. Server Finished

message encrypted with algorithms in the cipher spec. contains HMAC number generated from MD5 and SHA1 from master secret, "server finished" string, and hash of all prior handshake message

Examining TLS ClientHello [Privacy] (TODO)

In the TLS record protocol, a ClientHello message sent to the server before connecting. the message get sent unencrypted because this message and other in the handshake are used to get the encryption and compression method for future communication.

This message can be used to fingerprint your device and browser, even before sending the first HTTP request. Some data like the TLS record version, cipher suite list, compression method list, and Hello Extensions that are used or not may be unique for different browser or software on different device. Which allow for fingerprinting.

(todo: write a detailed explanation about how this is done)

Additional links

The Secure Sockets Layer (SSL) Protocol Version 3.0: https://datatracker.ietf.org/doc/html/rfc6101

The Transport Layer Security (TLS) Protocol Version 1.2: https://datatracker.ietf.org/doc/html/rfc5246

The Transport Layer Security (TLS) Protocol Version 1.3: https://datatracker.ietf.org/doc/html/rfc8446

Information missing here

Alert Protocol

Initialization Vector

Stream Ciphers

Block Ciphers

AEAD Ciphers

HTTP/1.1 Webclient [Util]

How to request to server with HTTP/1.1

A request pipeline can be done by having separate thread for both sending and receving from the socket.

Common headers

Higher q value is higher preference for an option. q value can be from 0 to 1 and use 1 to 3 decimal places. The q value only applies to the preceding option

Accept

mime types that receiver expecting. comma separates, semicolon for q value. Implied q value is 1.0. Mime type can be */* (any), type/* (any subtype of type) or type/subtype. More specific mime type with same q values are preceding less specific types with same q value.

Accept-Encoding

Compressions that receiver can use, but not necessary. only compression are br, compress, deflate, gzip, identity, and * (any) Comma separates, semicolon for q value. Implied q value is 1.0.

Accept-Language

language requested. comma separated, * for any, semicolon for q value. implied q value is 1.0

Authorization

Basic with base64 username:password. This gets sent when a url has the protocol://username:password@domain:port/ or when authorization is required from a response, but server may not use basic in that case. other authorization exist. port is optional and is default of 80 for http or 443 for https

Connection

close, keep-alive, or upgrade. HTTP/1.1 Connection header default is keep-alive compared to HTTP/1.0 which is close. keep-alive will reuse the socket for requests. close will close the socket once the request is done

Content-Length

length of body of request or response

Content-Type

mime type of body of request or response

Cookie

cookie=value; separated by semicolon

Host

Server domain or IP and optional port if port not 80 or 443. can be used for a proxy service

If-Modified-Since

check if modified since a date.

If-None-Match

check ETag for cache. can be multiple ETag with comma separating

Origin

Domain of current page.

Referer

full path of current page

Upgrade

The new protocol request by the client, multiple protocols can be comma separated. Server may reject

User-Agent

Browser version and maybe operating system. Can also have webrenderer

Common methods

GET

receive a resource. Should not modify state of resource on server. do not include sensitive data in the query, for that you need POST

HEAD

Perform like a GET request, but server will only send headers. Useful for ETag check

POST

for forms or for send large data, like a file with multipart/form-data mimetype. may change the resource. forms use application/x-www-form-urlencoded if not specified

OPTIONS

Used to determine the request methods that the server support. also be used to query the access control for cross origin requesting

Sample requests

GET / HTTP/1.1\r\n
Host: example.com\r\n
\r\n

simple GET request, using Host header. Host header must be present in all requests. Host header shows the domain example.com and no port, which means the port should be 80 if using http or 443 if using https. A GET request will not have a body, so Content-Length: 0 is not needed.


GET /index.php?show_page=100 HTTP/1.1\r\n
Accept: */*\r\n
Accept-Encoding: compress, gzip\r\n
Content-Length: 0\r\n
Host: example.com:1111\r\n
If-None-Match: "05aafa1b"\r\n
User-Agent: Mozilla/5.0 (OS 1.0)\r\n
\r\n

A request with a query. The file is index.php and query is show_page=100. Accept: */* means the client will accept any mime type response from the server. the encoding the client wants is compress or gzip, with both implied q=1.0 which is equal preference, but the server does not need to compress. Host: example.com:1111 shows the domain example.com and port 1111. If-None-Match will use an ETag for the page to only get the page when the ETag is not found on the server for the request page.


POST / HTTP/1.1\r\n
Accept: text/html, application/json; q=0.5\r\n
Accept-Language: zh, de-DE, *; q=0.575\r\n
Content-Length: 9\r\n
Content-Type: application/x-www-form-urlencoded\r\n
Host: example.com\r\n
\r\n
form=data

A POST request with some form data. The form uses POST method and post to /. Accept has text/html with implied q=1.0 and application/json with q=0.5, which mean text/html has a higher preference. Accept-Language has zh and de-DE languages both with implied q=1.0 equal preference. zh and de are base language. A sub language (ex: DE) can come after base language, but it is optional. zh language has no sub language in this request. *; q=0.575 means all other languages have a q value of 0.575 and is an example of a q value with 3 decimal places which is the maximum of decimal places. Content-Length now is the number of bytes in the body. the Content-Type is the default for forms. In the body of the request, the form field with the name 'form' has the value 'data'.


POST /?a=1&b=2 HTTP/1.1\r\n
Content-Type: multipart/form-data; boundary=boundary\r\n
Host: example.com\r\n
\r\n
--boundary\r\n
Content-Disposition: form-data; name="text"\r\n
\r\n
value\r\n
--boundary\r\n
Content-Disposition: form-data; name="file"; filename="file"\r\n
\r\n
file... (-snip-)
\r\n
--boundary--

A POST request with some form data and a query. This form also uses POST method and posts to / with the query a=1 and b=2. Queries are separated with & and are encoded with percent signs and hexadecimal for any characters like spaces, ?, =, & and others. this Content-Type uses multipart/form-data with a boundary of boundary, so the text --boundary-- must not appear after a CRLF in any of the data except at end. The boundary may be in quote or not. In the body of the request, the boundary after -- is there and then CRLF. Then Content-Disposition header is used for data about the form field. other headers may follow this. It ends with CRLFCRLF like the request headers. Then the value with another CRLF which is not part of the value. if the request body is done, a -- is put and then the boundary and then another --. If not done, then the same process repeats.

This body shows a form with a text field with the name text and the value value and a file field with the name file which has filename file on the disk and then data of the file.

Additional links

Hypertext Transfer Protocol -- HTTP/1.1: https://datatracker.ietf.org/doc/html/rfc2616

Information missing here

Range requests

Other Authorization

Other headers

HTTP/1.1 Webserver [Util]

Sample response and header information. same pipeline as client can be done

Common headers

Content-Encoding

If compression applied. multiple can be applied, separate by comma in order of the applied. Only compression are br, compress, deflate, and gzip.

Content-Type

Same meaning as client.

Content-Length

Same meaning as client. does not appear if Transfer-Encoding: chunked

Connection

Same meaning as client.

Date

The date and time in GMT.

ETag

value to indicate to client if modified or not

Location

the next url to go to for redirects. Starts on current path, unless begins with forward slash or full url

Server

description of server and version.

Set-Cookie

Set a cookie=value. Can limit to a domain or path. Can set to expire or force expire if date is before current date. Can be httponly (javascript can not use the cookie).

Transfer-Encoding

If compression applied or if chunked. both can apply, separate with comma. Allowed from chunked, compress, deflate, gzip, and identity. chunked uses hexadecimal number as size of the chunk, follow by CRLF, then chunk data, then CRLF. chunked ends when 0 length chunk followed by CRLFCRLF. Headers may come after the zero length chunk after the first CRLF. Transfer-Encoding: chunked must be the last thing done to the body

Common statuses

101 Switching Protocols

200 OK

206 Partial Content

301 Moved Permanently

302 Found

304 Not Modified

403 Forbidden

404 Not Found

500 Internal Server Error

Sample responses

HTTP/1.1 101 Switching Protocols\r\n
Connection: upgrade\r\n
Upgrade: HTTP/2.0\r\n
\r\n

When the server accepts the client request to switch protocols or the server wants to upgrade to process the request.


HTTP/1.1 200 OK\r\n
Connection: keep-alive\r\n
Content-Length: 100\r\n
Content-Type: text/html; charset=utf-8\r\n
Date: Mon, 05 Jul 2021 00:43:22 GMT\r\n
ETag: W/"1111"\r\n
Server: server/1.2.3 (Unix)\r\n
Set-Cookie: cookie=value\r\n
\r\n
<!DOCTYPE HTML><html>. . . (-snip-). . . .</html>

A response with a 100 byte html in utf-8 encoding. the connection will stay open when after send to client. The Date header shows the date. the ETag header gives the client a value to check if page change. The Server header shows details about the server. the Set-Cookie header wants the client to set the cookie cookie with the value value and is not httponly.


HTTP/1.1 200 OK\r\n
Content-Type: application/json\r\n
Transfer-Encoding: chunked\r\n
\r\n
6\r\n
{"mani\r\n
E\r\n
fest_version":\r\n
2\r\n
4}\r\n
0\r\n
\r\n

a application/json response with unknown charset that is sent using Transfer-Encoding: chunked. The Connection: keep-alive is missing, but the connection will stay open because of HTTP/1.1. The first chunk is 6 bytes, the second chunk is 14 bytes, the third thunk is 2 bytes, and the last chunk is 0 bytes which meanings no more chunks is in the response.


HTTP/1.1 301 Moved Permanently\r\n
Connection: close\r\n
Location: /index.html\r\n
\r\n

A response that will redirect the client to /index.html on the same domain and close the connection.

Additional links

Hypertext Transfer Protocol -- HTTP/1.1: https://datatracker.ietf.org/doc/html/rfc2616

Information missing here

HSTS (Strict-Transport-Security)

CSP (Content-Security-Policy)

CORS (Access-Control-Allow-Origin, Access-Control-Allow-Methods, ...)

Other headers

Other statuses

418 I'm a teapot (https://datatracker.ietf.org/doc/html/rfc2324#section-2.3.2)

HTTP proxy service [Piracy]

Once you have the webclient and webserver, you can combine them to make a proxy service. Proxy services are great for piracy because then you are able to intercept the very HTTP traffic and then do analyse on them to save specific parts to a file (ex: a video on the web loads m3u8, the proxy can capture that and save files if the links in the m3u8 go through the proxy. For encrypted video stream in the body of HTTP decrypted by the client, the HTTP proxy will be of no use unless you wish to document the encrypted stream on the file system. this does not mean the idea of proxy is dead, because you just need to proxy the decryption algorithm instead, which may use a different protocol and may be very hard depending if hardware is involved)

/etc/hosts

Using /etc/hosts can make the request go to 127.0.0.1 which the proxy can listen on. Windows use %windir%\System32\drivers\etc\hosts

Example

127.0.0.1 example.com

in /etc/hosts will make example.com domain go to 127.0.0.1

Sample proxy

proxy = tcp stream socket listening on port 80

client = socket from proxy accept
server = tcp stream socket
connect server to example.com on port 80

receive from client and send to server
receive from server and send to client

Using a thread pool, you can accept many client requests instead of one at a time. Also, make sure to check if HTTP/1.0 or HTTP/1.1 and the Connection header to see if the socket should close. HTTP/1.0 closes if no Connection header. HTTP/1.1 do keep-alive if no Connection header.

Extension

If you wanting to proxy multiple domains, add them to the /etc/hosts. Change the proxy so that it doesnt connect to the server until the client is done with HTTP headers. With http headers copy, you can figure out the server that you should connect to in Host header. send the copy and then continue receive from client and send to server.

Using openssl [Util]

Generate an RSA private key

$ openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out rsa.key

(sidenote: rsa2048 is default)

if you wanting RSA1024 or RSA4096, change the rsa_keygen_bits:2048 to rsa_keygen_bits:1024 or rsa_keygen_bits:4096

OR use

$ openssl genrsa -out rsa.key 2048

replace 2048 with number of bit for the key

To encrypt the private key file with a password make a file called password in the directory and edit it so that it is the password on one line and ends with new line

$ openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out rsa.key -aes256 -pass file:password

OR use

$ openssl genrsa -out rsa.key -aes256 -passout file:password 2048

Replace aes256 with the cipher wanted to encrypt the key with


to view the key

$ openssl pkey -in rsa.key -text -noout

or the public part only

$ openssl pkey -in rsa.key -text_pub -noout

To view the key with password

$ openssl pkey -in rsa.key -passin file:password -text -noout

or the public part only

$ openssl pkey -in rsa.key -passin file:password -text_pub -noout

Generate an X509 root certificate

all root certs are self signed

First you need a certificate request. use private key for -key

$ openssl req -new -key rsa.key -out cert.req

to view

$ openssl req -in cert.req -text -noout

Then using the cert request, do

$ openssl x509 -req -in cert.req -signkey rsa.key -out root.cert

then, to view the cert, use

$ openssl x509 -in root.cert -text -noout

Generate a X509v3 root certificate

To do this, you need to specify the extensions in the cnf file. You can copy the authority key id from the signer certificate. the serial number of X509 certificate is arbitrary and chosen by the certificate authority.

A better way to generate the X509 root certificate

$ openssl req -new -x509 -key rsa.key -out root.cert

will generate a X509 root certificate with x509v3 extensions but cnf file is needed to control the extensions

sample req config

[req]
default_bits=2048
default_keyfile=default.key
distinguished_name=dn
attributes=req_attributes
prompt=no
req_extensions=v3

[req_attributes]

[dn]
C=NZ
ST=State
L=Locality
O=Organization
OU=Organizational Unit
CN=Common Name
[email protected]

[v3]
subjectKeyIdentifier=hash
basicConstraints=critical,CA:true

C is the 2 letters Country Code (ex: NZ), default.key file do not need to exist. only country name and common name are required in the dn section. Others like state, locality, etc are optionals

The subject key identifier can be set using a hex string if wanted

subjectKeyIdentifier="111122223333444455556666777788889999AAAA"

generate the certificate request

$ openssl req -new -key rsa.key -config config.cnf -out cert.req

Sample ca config

[ca]
default_ca=CA_default

[CA_default]
dir=.
database=$dir/index.txt
new_certs_dir=$dir/newcerts
certificate=$dir/ca.cert
serial=$dir/serial
private_key=$dir/ca.key
default_startdate=YYMMDDHHMMSSZ
default_enddate=YYMMDDHHMMSSZ
default_md=sha256
policy=policy_any
x509_extensions=v3

[policy_any]
countryName=supplied
stateOrProvinceName=optional
organizationName=optional
organizationalUnitName=optional
commonName=supplied
emailAddress=optional

[v3]
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid
basicConstraints=critical,CA:true

Create a empty file called index.txt and make a directory called newcerts. ca.cert and ca.key should be the certificate and private key for the ca. ca.key can be rsa.key for this because of self sign. Create the serial file and put in "02" or a hex like "111122223333444455556666777788889999AAAA" or add rand_serial=yes after the serial=$dir/serial line in cnf for a random serial which will not use serial file. but the serial file must exist even if not used by rand_serial=yes. default_md can be anything from the $ openssl dgst -list Change the default_startdate and default_enddate (ex: 210731123456Z is 2021-07-31 12:34:56 GMT)

then make the cert

$ openssl ca -selfsign -key rsa.key -in cert.req -config config.cnf -out root.cert

X509v3 Extensions

Edit the v3 section

Sample Basic Constraints

basicConstraints=CA:true
basicConstraints=CA:false
basicConstraints=critical,CA:true,pathlen:0

pathlen is the number of intermediate certificate that can be in the chain after this certificate. If it is 0 the certificate can only sign the server certificate

Sample Key Usage

can be any comma separated values from digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment, keyAgreement, keyCertSign, cRLSign, encipherOnly, decipherOnly. Can have critical

keyUsage=digitalSignature
keyUsage=critical,dataEncipherment,cRLSign

Sample Extended Key Usage

can be any comma separated values from serverAuth, clientAuth, codeSigning, emailProtection, timeStamping, OCSPSigning, ipsecIKE, msCodeInd, msCodeCom, msCTLSign, msEFS. Can have critical

extendedKeyUsage=emailProtection
extendedKeyUsage=serverAuth,clientAuth
extendedKeyUsage=critical,serverAuth,clientAuth

Sample Certifiate Policies

can be any comma separated values of policy oid. Can have policy sections but not shown here

certificatePolicies=1.2.3.4
certificatePolicies=1.2.3.4,1.1.1.1.1

Sample Alternate DNS

subjectAltName=@alt_names

[alt_names]
DNS.1=127.0.0.1
DNS.2=example.com
DNS.3=*.example.com

Generate a X509 certificate chain

In order to verify a certificate, the hash of the certificate is taken and signed by the signer cert's private key. using the same hash algorithm, and the public key of the signer cert, it checks if the signature matches (same hash).

if previous step from the guide was done, rename rsa.key to root.key. If not, this step needs a file called root.cert and root.key which is the root self sign certificate and the root private key

Generate private keys called intermediate.key and server.key and make requests for them. Make distinguished_name section something else for each request. for server.req use common name for domain name or use alternate dns extension. To create intermediate.cert, change the certificate=$dir/ca.cert to certificate=$dir/root.cert and change private_key=$dir/ca.key to private_key=$dir/root.key. change the dates and default_md and extensions if desire then

$ openssl ca -in intermediate.req -config config.cnf -out intermediate.cert

Then change certificate=$dir/root.cert to certificate=$dir/intermediate.cert and change private_key=$dir/root.key to private_key=$dir/intermediate.key. change the dates and default_md and extensions if desire then

$ openssl ca -in server.req -config config.cnf -out server.cert

Congratulations you now have a certificate chain!

Additional links

genpkey - generate a private key: https://man.archlinux.org/man/genpkey.1ssl

pkey - public or private key processing tool: https://man.archlinux.org/man/pkey.1ssl

config - OpenSSL CONF library configuration files: https://man.archlinux.org/man/config.5ssl.en

req - PKCS#10 certificate request and certificate generating utility: https://man.archlinux.org/man/req.1ssl

x509 - Certificate display and signing utility: https://man.archlinux.org/man/x509.1ssl

x509v3_config - X509 V3 certificate extension configuration format: https://man.archlinux.org/man/core/openssl/x509v3_config.5ssl.en

ca - sample minimal CA application: https://man.archlinux.org/man/ca.1ssl

Information missing here

Other Certificate Policy

Other Subject Alternative Names

CRL

CTL

SCT

OCSP

Related

LibreSSL

GnuPG

GnuTLS

HTTPS proxy service [Piracy]

Using information from openssl to create certificate, you only need to change sockets to ssl wrapped sockets which have a context of your certificate chain and server private key. This is really easy if you already have your code for your own HTTP proxy service. The ssl contexts will do the encryption. Remember to install your proxy's root certificate in both the certificate store of the ssl context and the client.

Sample proxy

context = new ssl context
give this ssl context your server certificate private key
give this ssl context your server certificate chain or
the root certificate if there is no chain and the common
name is domain

defaultContext = default ssl context

proxy = tcp stream socket
wrap proxy socket in the ssl context then
make proxy listen on port 443

client = socket from proxy accept
server = tcp stream socket
wrap server in the default ssl context then
connect server to example.com on port 443

receive from client and send to server
receive from server and send to client

Same advice and extension as the HTTP proxy, but just use ssl contexts. To proxy multiple domains, it may be needed to generate many contexts and certificates or use alternate DNS name with a single certificate.

HTTP/2 [Util] [Extension] (TODO)

HTTP/2 is released on 2015. It is the same fundamental design as HTTP/1.1 (use tcp/ip and tls, but for HTTP/2 tls is like a requirement and not optional). HTTP/2 no longer uses request pipeline as HTTP/1.1 (but replaces with multiplexing). HTTP/2 marks Transfer-Encoding: chunked HTTP header invalid. To upgrade to HTTP/2 a client sends header with Connection: upgrade and Upgrade: HTTP/2.0 (todo: explain HTTP/2 frames)

HTTP/3 [Util] [Extension] (TODO)

HTTP/3 is released on 2019. HTTP/3 uses HTTP/2 api, but over udp instead of tcp. They made new protocol called QUIC which pushes packet resending and integrity information to application level instead of kernel level. This eliminates head of line blocking in HTTP/2 (todo: explain QUIC)

WebSockets (TODO)

The client will send a HTTP request with Connection: upgrade and Upgrade: websocket (todo: examine the protocol and see if it can be used to intercept media)

Insecure WebSockets (TODO)

(todo: examine the protcol)

Secure WebSockets (TODO)

secure websockets operate on TLS (todo: examine the protcol)

Additional links

The WebSocket Protocol: https://datatracker.ietf.org/doc/html/rfc6455

Examining DirectShow WindowsAPI [Privacy] (TODO)

(todo: examine if it is possible to create a fake webcam feed using C++ and DirectShow WinAPI for privacy reasons)

Examining the ZIP File format [Steganography] [Privacy] (TODO)

The zip file format has an extra field in local file headers that may be used for metadata. the zip file format version made by can fingerprint (ex: 0x1e03 in the version made by is a little endian 2 byte integer specifying that the zip file was created on *nix and the version support 3.0 of the zip standard). the external file attributes may be used for fingerprinting.

Because the central directory header of a zip file is at the end, it easy to append a zip file to like an image file (very basic steganography). (todo: how to change file creation time using a zip file) (todo: more details)

Additional links

.ZIP File Format Specification 6.3.3: https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT

Examining the PNG File format [Steganography] [Privacy]

The png starts with hex 89504E470D0A1A0A and then png chunks come after. integers are big endian. Each chunk contains the 4 byte unsigned integer for the length of the data, the chunk type is the 4 byte ascii for the chunk type, the data, and then the CRC check for the data includes chunk type and chunk data but not the length. Each 4 byte ascii can be checked with bitwise and of 32 (&32) to make additional info about headers.

Because of ascii, if it lowercase, then the bitwise and is 32 and 0 for uppercase. if the first letter is uppercase, then the chunk is required in the image. it is lowercase then the chunk is not. If the second letter is uppercase, then the chunk type is public. If it is lowercase then the chunk type is private. the third letter case is reserved. if the fourth letter is uppercase, then the chunk should not be copy and if lowercase the chunk can be copy.

The CRC is done using the number EDB88320 for table calculation (read this hexadecimal number in binary from right to left to get the polynomial but without 32 power)

IHDR must be first chunk and IEND must be the last chunk. for the simplest image, one or more IDAT come after IHDR and those are the only 3 chunk type in the image. all IDAT are compressed using deflate

IHDR has a length of 13 and add chunk type ascii for hex 0000000D49484452. The first 4 byte are the image width, the next 4 bytes is the image height. The 5 next are all 1 byte which are the bit depth, the color type, the compression method, the filter method, and interlace method.

The example image is using bit depth 8, color type 6 (see below), compression method 0 (deflate), filter method 0 (null), interlace method 0 (null).

There are only 5 type of png image

Truecolor and alpha = 6

image has RGB values and alpha value

Grayscale and alpha = 4

image has gray value and alpha value

Truecolor = 2

image has RGB values

Grayscale = 0

image has gray value

Indexed = 3

image use a table to index at most 256 distinct colors. this table must be the PLTE chunk and only appear before any IDAT

Every pixel is made from channels with a sample depth. In the png, the channels have the same sample depth. The pixels come from samples of the channels or the index if indexed color. The bit depth represents the number of bits per sample or the bits in every table color if indexed color.

the image can use Adam7 interlace which make different images from the original pixels for each group of pixel in a pattern, each scanline is made from the pattern and represent that pixel group. Then all scanlines from top to bottom are turned into bytes, with the left pixel first and the right pixel last. it can use filter before compression to help the deflate compress. Then it is compressed stream and placed into image data chunks.

after the IHDR CRC comes the IDAT chunk hex ????????49444154. replace ? with length of chunk in hex big endian. Some programs limit IDAT length. Inside the IDAT chunk is deflate compressed data. The first byte is the compression method which is 08 (deflate), the next byte is the flags which is 1D (00011101 in binary). The first two bits 00 indicates compression with the fastest algorithm. the next bit 0 indicates no dictionary. The next bits 11101 are the check value for the two bytes where the first byte * 256 + the entire flags byte must be multiple of 31.

8*256 + 29 = 2048 + 29 = 2077 which is multiple of 31

then the compressed data is next. The byte will be the deflate method header and it 01 (binary 00000001). because last bit is 1, this is the last compressed block. The two bits to left of the 1 are the compression type which is 00 for no compression. other compression types are 01 for fixed Huffman, 10 for dynamic Huffman. 11 is error. Because of compression type no compression, next two byte is the length. This is the first byte + 256 * second byte. then next two byte is the one complement of the length.

After this is the actual compressed image data (but this using type 00 for no compression) which mean only length from deflate header bytes after that. Then is the ADLER-32 of the uncompressed data. Then is the CRC of the image chunk.

How compute the ADLER-32 can be found in the RFC1950 and CRC32 can be found in the W3 specification

After this a 0 length IEND header 0000000049454E44AE426082 ends the image.

Steganography

  1. The lower bit of selection of pixel sample can be change to make a binary message in the file which wont be visible to a human eye because it only affect lower bit. but only do this to image that is colorful

  2. image is truecolor with alpha (type 6) and since the bit depth is 8, it fits ascii in the RGBA pattern. If G and A are 0, it is like wide string where R and B are hold the ascii. Because A are 0 in every pixel, the image look transparent but actual contains some data

download the image as image.png and try using strings in linux or open with notepad on windows

$ strings image.png

$ strings -e l image.png

notice the different result between the outputs.

  1. Based on image chunks and not pixels there are still ways to do steganography. Example: send a lot of png, but make the number of IDAT chunks mean something. or use the PLTE chunk and other metadata chunks since it can has lots of bytes. Or make the specific chunk ordering different. there are a lot of way to do this based on only chunk alone

  2. Or use pixels and chunks both for the steganography

Fingerprinting

It may be possible but very unlikely (i think) to discover the program used to make the picture by checking chunk ordering, chunk present and chunk count (ex: some program may include sRGB and gAMA chunk while other does not), private chunks, how and when the compression is done, color palette, color type, or other chunk metadata

some chunk like a text chunk can have metadata or compressed metadata for a keyword. Even if program that made picture is not know, this text metadata content may fingerprint.

Additional links

Portable Network Graphics (PNG) Specification (Second Edition): https://www.w3.org/TR/PNG/

ZLIB Compressed Data Format Specification version 3.3: https://datatracker.ietf.org/doc/html/rfc1950

Adam7 algorithm: https://en.wikipedia.org/wiki/Adam7_algorithm

Information missing here

Interlaced image

Other chunks (cHRM, gAMA, iCCP, sBIT, sRGB, bKGD, hIST, tRNS, pHYs, sPLT, tIME, iTXt, tEXt, zTXt) and their purpose and order in file

Filtered scanline functions

Changing File times [Privacy] (TODO)

Windows (TODO)

The creation time can be changed using a zip file (see: Examining the ZIP File format (TODO)), but you need to use MSDOS time and date (both 16bit integer) (todo: what is MSDOS time and date) Otherwise, you need the SystemTimeToFileTime and SetFileTime functions of the windows api

Linux

You can use stat to see the info for a file

$ stat somefile
  File: somefile
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 801h/2049d	Inode: 6160770     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    user)   Gid: ( 1000/    user)
Access: 2021-07-14 06:38:12.138649334 +0000
Modify: 2021-07-14 06:38:12.138649334 +0000
Change: 2021-07-14 06:38:12.138649334 +0000
 Birth: -

we are interested in the access, modify, and change times

You can use touch to set time info for a file. you can only have full control over access and modify. change times are updated by the kernel to the current time. touch -a to change access. touch -m to change modify.

$ touch -a -d "2000-08-08 12:34:56.1234123" somefile
$ touch -m -d "2008-04-09 11:22:33.4444123" somefile
$ stat somefile
  File: somefile
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 801h/2049d	Inode: 6160770     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    user)   Gid: ( 1000/    user)
Access: 2000-08-08 12:34:56.123412300 +0000
Modify: 2008-04-09 11:22:33.444412300 +0000
Change: 2021-07-14 06:49:42.294649334 +0000
 Birth: -

The resolution of time may not be as precise because of the used file system.

(i do not recommend) to change the change time, you need to either patch the kernel for your system to stop automatically setting the change time, or you need to open the entire disk at a very low level and find and change data of the file's inode (none of these options are recommend by me!)

DEFLATE [Util] (TODO)

Huffman codes are prefix codes which mean that it's a set with no element being a prefix of any other. This variable length coding can be done based on frequency, which means that the shorter lengths get assigned to characters that appear more often. there is also backreference and length pairs that can repeat strings up to 32K before in the buffer. The Huffman tree can be reconstructed from a set of integers that are the bit lengths and count of bit lengths in a lexicographic ordering. (todo: describe the compression algorithm)

Additional links

DEFLATE Compressed Data Format Specification version 1.3: https://datatracker.ietf.org/doc/html/rfc1951

Make a Firefox Addon [Util]

A Firefox addon can useful to add javascript into websites or to intercept HTTP requests. Also, there are many other types of Firefox addons to make. for each of the addons in the guide, make a directory if seems useful.

Content Scripts

This addon will add javascript into websites.

File: manifest.json

{"manifest_version":2,
"name":"the name",
"version":"1.0",
"description":"the description",
"icons":{"32":"image.png"},
"content_scripts":[{
    "matches":["https://example.com/"],
    "js":["script.js"]
}]}

File: image.png

32x32 png image

File: script.js

console.log('this is script')

Now install temporary addon and then go to https://example.com and open console. Try changing matches array in manifest.json by adding new urls or using wildcard like https://example.com/* or *://*/*. then see what happens when you visit urls. Change the script.js file to any javascript.

Intercept HTTP

This addon will intercept HTTP.

File: manifest.json

{"manifest_version":2,
"name":"the name",
"version":"1.0",
"description":"the description",
"icons":{"32":"image.png"},
"permissions":["*://*/*","webRequest","webRequestBlocking"],
"background":{"scripts":["script.js"]}}

(sidenote: you can both use content_scripts and background in manifest.json)

File: image.png

32x32 png image

File: script.js

function beforeRequest(detail){
if(!detail.url.indexOf('https://example.net')){
let f=browser.webRequest.filterResponseData(detail.requestId),
t=new TextEncoder();f.onstart=function(e){
f.write(t.encode('overwritten'));
f.close();f.disconnect()};
}
if(!detail.url.indexOf('https://example.com'))return{cancel:true};
return{};
}
browser.webRequest.onBeforeRequest.addListener(
beforeRequest,{urls:['*://*/*']},['blocking']);

Install temporary addon then this script will cancel all requests starting with https://example.com which can see if open console then go to network tab. Also will overwrite all requests start with https://example.net with text 'overwritten', try to open a new tab and go to https://example.com and then try https://example.net to see

Install Temporary Addon

Visit about:debugging then go to This Firefox link. Then load the addon by selecting any file (ex: manifest.json) in the addon directory. if using private mode, remember to visit about:addons to enable it for private mode

Additional links

Browser Extensions - Mozilla | MDN: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions

webRequest.onBeforeRequest - Mozilla | MDN: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/onBeforeRequest

webRequest.StreamFilter - Mozilla | MDN: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/StreamFilter

Crack your First Software [RE] (TODO)

some tools: IDA, radare2 (linux), gdb (linux), Hopper, ghidra, ollydbg (windows, obsolete), x64dbg, windbg (todo: find software on web that would be easy for a beginner. go through thoughts and explain what to do to crack the software) (note: this is for beginner, do not discuss crack DRM here)

Cracking Java Software [RE] (TODO)

some tools: jdgui, java decompiler (todo: find software on web that would be easy for beginning)

Cracking .NET Software [RE] (TODO)

some tools: ILSpy (see source), de4dot (deobfuscate) (todo: get software on web that would be easy for beginner)

Making your First keygen [RE] (TODO)

(todo: find easy software to make a keygen for, then describe thought process and how it is can be done)

Cracking DRM [RE] (TODO)

some tools: DeDRM

handbrake with some libraries can also do it

Always online DRM can be bypassed if a server is programmed for it. /etc/hosts will be useful like in HTTP or HTTPS proxy

(todo: more research and aggregate and post back here)


Source code

```
     ░ ▓ ▒                      ░          ░                          ▒ ▓ ░     
     ░ ▓                         ░  ███▄  ░                             ▓ ░     
     ░                             ██∙███                                 ░     
          █▄ ▄█│  ▄▄█████│       ░ ██  ██ ░   │███   │█████▄▄  ▀█   ▄           
     ████│██▄██│ ██═════▀█│ █████│  ▀█▀█   ░│██  │█ │█▀═════██ │██ │█ │███▄     
     █│   █│█ █│██       ██│  █│   █▌   ▐█ │██     │██       ██│█│█│█│█▀ ▐▀     
     ██▌  █│  █│██│      ██│  █│    █████  │██  │█ │██      │██│█│█│█│█▄        
     █│   █│  █│██│      ██│  ▐│     ███▀   │████  │██      │██│█ │██ │██▄      
     ▀███│▀   █│ ██     ██│          ███            │██     ██ │█  │██  │█      
        ▀         █████▀▀  ▌         ██            ▐  ▀▀█████       ▀ ▀██▀      
     ░             ▄▄    ▄▀           ▀█            ▀▄    ▄▄              ░     
     ░ ▓            ▀▀▀█▀              ▀              ▀█▀▀▀             ▓ ░     
     ░ ▓ ▒              ▀▀▄▄▄▄▄  . EMOTICONS .  ▄▄▄▄▄▀▀               ▒ ▓ ░     
     ░ ▓ ▒                   ▀▀▀▓███▓▄▄█▄▄▓███▓▀▀▀                    ▒ ▓ ░     
     ░ ▓ ▒                         ▀▀▀▀█▀▀▀▀                          ▒ ▓ ░     
     ░ ▓ ▒        :                                          :        ▒ ▓ ░     
     ░ ▓ ▒       :~~~~~~~~~~~~. PROUDLY PRESENTS .~~~~~~~~~~~~:       ▒ ▓ ░     
     ░ ▓ ▒        :                                          :        ▒ ▓ ░     
     ░ ▓ ▒                      emoticons-guides                      ▒ ▓ ░     
       ▓ ▒                 These are some guides that                 ▒ ▓       
         ▒                are related to computer and                 ▒         
                                   interests.                                   
                                                                                
                             Take a look around! :)                             
                  :                                          :                  
                 :~~~~~~~~~~~~~~~~~. TAGS .~~~~~~~~~~~~~~~~~~~:                 
                  :                                          :                  
                                   Extension                                    
                                     Piracy                                     
                                    Privacy                                     
                                       RE                                       
                                 Steganography                                  
                                      Util                                      
     ░            :                                          :            ░     
     ░ ▓         :~~~~~~~~~~~~~~~. WRITTEN BY .~~~~~~~~~~~~~~~:         ▓ ░      
     ░ ▓ ▒        :                                          :        ▒ ▓ ░     
     ░ ▓ ▒                        /u/emoticons                        ▒ ▓ ░     
     ░ ▓ ▒        :                                          :        ▒ ▓ ░     
     ░ ▓ ▒       :~~~~~~~~~~~~~~~. DISCLAIMER .~~~~~~~~~~~~~~~:       ▒ ▓ ░      
     ░ ▓ ▒        :                                          :        ▒ ▓ ░     
     ░ ▓ ▒         emoticons-guides is very basic information         ▒ ▓ ░     
     ░ ▓ ▒                 and maybe not all accurate                 ▒ ▓ ░     
```

TLS Record Protocol
===
it is used to verify the identity of servers with asymmetric key and to make a symmetric key exchange from master secrets. The master secret is split up into keys, MAC keys and if require IV keys for the client and server both. (example: using diffie hellman from client and server shared pre master secrets) bytes are big endian. also can do Alert Protocol and Change Cipher Spec Protocol. As of this day, TLS 1.3 is the most up to date of this protocol. TLS 1.3 simplifies handshake by a lot. TLS comes from SSL, but TLS is more better with more update encryption.

The Record
---
The record just a wrapper for TLSCompressed (same format as TLSPlaintext) structure, or a TLSCiphertext. the first byte shows the content type. The next byte show the major protocol version. the next byte show the minor protocol version. The next two show the length. Then follows the data in the record. this data is compressed using the algorithm from the handshake, but on starting handshake, there is no compression. Encryption is done after compression, depending on cipher spec. MAC is added to compressed records. MAC algorithm depend on cipher spec. MAC will be calculated before encryption. A record can have more than one message of the same content type or the partial content of single message.

Handshake TLS1.2
---

A handshake is done from a client first. and a handshake done on top of the record.

1. ClientHello and ServerHello

Client and server decide on the TLS version, cipher suite and compression used. client and server share random values. a session identify assigns to the client if not connection before. this is always a random session identity. If a client shows a previous session identifier to the server, it can continue a session because the cryptography can be expensive, a session will make sure the key exchanges only done once.

2. Server Certificate (optional)

server will send intermediate and its own certificate to verify identity. Intermediate can be zero or more. Can send the root certificate as well, but is not required. Server certificate will contain a server public key

3. Server ServerKeyExchange (optional)

server sending a public key used to make the pre master secret. Only sent if server has no certificate or server's certificate is used for only signing

4. Server CertificateRequest (optional)

If the server need to confirm client identity.

5. Server ServerHelloDone

This tells client that server hello and server part of key exchange is finish

6. Client Certificate (optional)

Client will send certificate to verify identity. Only if request by Server CertificateRequest. if client has no certificate, message sent with zero length in data. Certificate or certificate chain sent the same as if a server sending the chain. client certificate contains client public key

7. Client ClientKeyExchange

for rsa client sends a pre master secret, encrypted in server certificate public key.

8. Client CertificateVerify (optional)

If a client send a certificate, it is verified by the digital signature. Indicating the client does has the private key for its certificate. Only send if the certificate is not have fixed Diffie Hellman. a hash of all prior handshake message is sent

9. Client ChangeCipherSpec

uses a different content type in the record. notify server that next messages will compress and encrypt with the chosen cipher suite and compression

10. Client Finished

message encrypted with algorithms in the cipher spec. contains HMAC number generated from MD5 and SHA1 from master secret, "client finished" string, and hash of all prior handshake message

11. Server ChangeCipherSpec

Using a different content type in the record. notify client that next messages will compress and encrypt with the chosen cipher suite and compression

12. Server Finished

message encrypted with algorithms in the cipher spec. contains HMAC number generated from MD5 and SHA1 from master secret, "server finished" string, and hash of all prior handshake message

Examining TLS ClientHello [Privacy] (TODO)
---
In the TLS record protocol, a ClientHello message sent to the server before connecting.  the message get sent unencrypted because this message and other in the handshake are used to get the encryption and compression method for future communication.

This message can be used to fingerprint your device and browser, even before sending the first HTTP request. Some data like the TLS record version, cipher suite list, compression method list, and Hello Extensions that are used or not may be unique for different browser or software on different device. Which allow for fingerprinting.

(todo: write a detailed explanation about how this is done)

Additional links
---
The Secure Sockets Layer (SSL) Protocol Version 3.0: https://datatracker.ietf.org/doc/html/rfc6101

The Transport Layer Security (TLS) Protocol Version 1.2: https://datatracker.ietf.org/doc/html/rfc5246

The Transport Layer Security (TLS) Protocol Version 1.3: https://datatracker.ietf.org/doc/html/rfc8446

Information missing here
---

Alert Protocol

Initialization Vector

Stream Ciphers

Block Ciphers

AEAD Ciphers

HTTP/1.1 Webclient [Util]
===

How to request to server with HTTP/1.1

A request pipeline can be done by having separate thread for both sending and receving from the socket.

Common headers
---
Higher q value is higher preference for an option. q value can be from 0 to 1 and use 1 to 3 decimal places. The q value only applies to the preceding option

**Accept**

mime types that receiver expecting. comma separates, semicolon for q value. Implied q value is 1.0. Mime type can be \*/\* (any), type/\* (any subtype of type) or type/subtype. More specific mime type with same q values are preceding less specific types with same q value.

**Accept-Encoding**

Compressions that receiver can use, but not necessary. only compression are br, compress, deflate, gzip, identity, and \* (any) Comma separates, semicolon for q value. Implied q value is 1.0.

**Accept-Language**

language requested. comma separated, \* for any, semicolon for q value. implied q value is 1.0

**Authorization**

Basic with base64 username:password. This gets sent when a url has the protocol://username:password@domain:port/ or when authorization is required from a response, but server may not use basic in that case. other authorization exist. port is optional and is default of 80 for http or 443 for https

**Connection**

close, keep-alive, or upgrade. HTTP/1.1 `Connection` header default is `keep-alive` compared to HTTP/1.0 which is `close`. keep-alive will reuse the socket for requests. close will close the socket once the request is done

**Content-Length**

length of body of request or response

**Content-Type**

mime type of body of request or response

**Cookie**

cookie=value; separated by semicolon

**Host**

Server domain or IP and optional port if port not 80 or 443. can be used for a proxy service

**If-Modified-Since**

check if modified since a date.

**If-None-Match**

check ETag for cache. can be multiple ETag with comma separating

**Origin**

Domain of current page.

**Referer**

full path of current page

**Upgrade**

The new protocol request by the client, multiple protocols can be comma separated. Server may reject

**User-Agent**

Browser version and maybe operating system. Can also have webrenderer

Common methods
---
**GET**

receive a resource. Should not modify state of resource on server. do not include sensitive data in the query, for that you need POST

**HEAD**

Perform like a GET request, but server will only send headers. Useful for ETag check

**POST**

for forms or for send large data, like a file with multipart/form-data mimetype. may change the resource. forms use application/x-www-form-urlencoded if not specified

**OPTIONS**

Used to determine the request methods that the server support. also be used to query the access control for cross origin requesting

Sample requests
---
```
GET / HTTP/1.1\r\n
Host: example.com\r\n
\r\n
```

---

simple GET request, using Host header. Host header must be present in all requests. Host header shows the domain example.com and no port, which means the port should be 80 if using http or 443 if using https. A GET request will not have a body, so `Content-Length: 0` is not needed.

---

```
GET /index.php?show_page=100 HTTP/1.1\r\n
Accept: */*\r\n
Accept-Encoding: compress, gzip\r\n
Content-Length: 0\r\n
Host: example.com:1111\r\n
If-None-Match: "05aafa1b"\r\n
User-Agent: Mozilla/5.0 (OS 1.0)\r\n
\r\n
```

---

A request with a query. The file is index.php and query is `show_page=100`. `Accept: */*` means the client will accept any mime type response from the server. the encoding the client wants is compress or gzip, with both implied q=1.0 which is equal preference, but the server does not need to compress. `Host: example.com:1111` shows the domain example.com and port 1111. `If-None-Match` will use an ETag for the page to only get the page when the ETag is not found on the server for the request page.

---

```
POST / HTTP/1.1\r\n
Accept: text/html, application/json; q=0.5\r\n
Accept-Language: zh, de-DE, *; q=0.575\r\n
Content-Length: 9\r\n
Content-Type: application/x-www-form-urlencoded\r\n
Host: example.com\r\n
\r\n
form=data
```

---

A POST request with some form data. The form uses POST method and post to `/`. `Accept` has text/html with implied q=1.0 and application/json with q=0.5, which mean text/html has a higher preference. `Accept-Language` has zh and de-DE languages both with implied q=1.0 equal preference. zh and de are base language. A sub language (ex: DE) can come after base language, but it is optional. zh language has no sub language in this request. `*; q=0.575` means all other languages have a q value of 0.575 and is an example of a q value with 3 decimal places which is the maximum of decimal places. `Content-Length` now is the number of bytes in the body. the `Content-Type` is the default for forms. In the body of the request, the form field with the name 'form' has the value 'data'.

---

```
POST /?a=1&b=2 HTTP/1.1\r\n
Content-Type: multipart/form-data; boundary=boundary\r\n
Host: example.com\r\n
\r\n
--boundary\r\n
Content-Disposition: form-data; name="text"\r\n
\r\n
value\r\n
--boundary\r\n
Content-Disposition: form-data; name="file"; filename="file"\r\n
\r\n
file... (-snip-)
\r\n
--boundary--
```

---

A POST request with some form data and a query. This form also uses POST method and posts to `/` with the query `a=1` and `b=2`. Queries are separated with `&` and are encoded with percent signs and hexadecimal for any characters like spaces, `?`, `=`, `&` and others. this `Content-Type` uses `multipart/form-data` with a boundary of `boundary`, so the text `--boundary--` must not appear after a CRLF in any of the data except at end. The boundary may be in quote or not. In the body of the request, the boundary after `--` is there and then CRLF. Then `Content-Disposition` header is used for data about the form field. other headers may follow this. It ends with CRLFCRLF like the request headers. Then the value with another CRLF which is not part of the value. if the request body is done, a `--` is put and then the boundary and then another `--`. If not done, then the same process repeats.

This body shows a form with a text field with the name `text` and the value `value` and a file field with the name `file` which has filename `file` on the disk and then data of the file.


Additional links
---

Hypertext Transfer Protocol -- HTTP/1.1: https://datatracker.ietf.org/doc/html/rfc2616

Information missing here
---
Range requests

Other Authorization

Other headers

HTTP/1.1 Webserver [Util]
===
Sample response and header information. same pipeline as client can be done

Common headers
---
**Content-Encoding**

If compression applied. multiple can be applied, separate by comma in order of the applied. Only compression are br, compress, deflate, and gzip.

**Content-Type**

Same meaning as client.

**Content-Length**

Same meaning as client. does not appear if `Transfer-Encoding: chunked`

**Connection**

Same meaning as client.

**Date**

The date and time in GMT.

**ETag**

value to indicate to client if modified or not

**Location**

the next url to go to for redirects. Starts on current path, unless begins with forward slash or full url

**Server**

description of server and version.

**Set-Cookie**

Set a cookie=value. Can limit to a domain or path. Can set to expire or force expire if date is before current date. Can be httponly (javascript can not use the cookie).

**Transfer-Encoding**

If compression applied or if chunked. both can apply, separate with comma. Allowed from chunked, compress, deflate, gzip, and identity. chunked uses hexadecimal number as size of the chunk, follow by CRLF, then chunk data, then CRLF. chunked ends when 0 length chunk followed by CRLFCRLF. Headers may come after the zero length chunk after the first CRLF. `Transfer-Encoding: chunked` must be the last thing done to the body

Common statuses
---
101 Switching Protocols

200 OK

206 Partial Content

301 Moved Permanently

302 Found

304 Not Modified

403 Forbidden

404 Not Found

500 Internal Server Error

Sample responses
---
```
HTTP/1.1 101 Switching Protocols\r\n
Connection: upgrade\r\n
Upgrade: HTTP/2.0\r\n
\r\n
```

---

When the server accepts the client request to switch protocols or the server wants to upgrade to process the request.

---

```
HTTP/1.1 200 OK\r\n
Connection: keep-alive\r\n
Content-Length: 100\r\n
Content-Type: text/html; charset=utf-8\r\n
Date: Mon, 05 Jul 2021 00:43:22 GMT\r\n
ETag: W/"1111"\r\n
Server: server/1.2.3 (Unix)\r\n
Set-Cookie: cookie=value\r\n
\r\n
<!DOCTYPE HTML><html>. . . (-snip-). . . .</html>
```

---

A response with a 100 byte html in utf-8 encoding. the connection will stay open when after send to client. The `Date` header shows the date. the `ETag` header gives the client a value to check if page change. The `Server` header shows details about the server. the `Set-Cookie` header wants the client to set the cookie `cookie` with the value `value` and is not httponly.

---

```
HTTP/1.1 200 OK\r\n
Content-Type: application/json\r\n
Transfer-Encoding: chunked\r\n
\r\n
6\r\n
{"mani\r\n
E\r\n
fest_version":\r\n
2\r\n
4}\r\n
0\r\n
\r\n
```

---

a `application/json` response with unknown charset that is sent using `Transfer-Encoding: chunked`. The `Connection: keep-alive` is missing, but the connection will stay open because of HTTP/1.1. The first chunk is 6 bytes, the second chunk is 14 bytes, the third thunk is 2 bytes, and the last chunk is 0 bytes which meanings no more chunks is in the response.

---

```
HTTP/1.1 301 Moved Permanently\r\n
Connection: close\r\n
Location: /index.html\r\n
\r\n
```

---

A response that will redirect the client to `/index.html` on the same domain and close the connection.

Additional links
---

Hypertext Transfer Protocol -- HTTP/1.1: https://datatracker.ietf.org/doc/html/rfc2616

Information missing here
---
HSTS (Strict-Transport-Security)

CSP (Content-Security-Policy)

CORS (Access-Control-Allow-Origin, Access-Control-Allow-Methods, ...)

Other headers

Other statuses

418 I'm a teapot ([https://datatracker.ietf.org/doc/html/rfc2324#section-2.3.2](https://datatracker.ietf.org/doc/html/rfc2324#section-2.3.2))

HTTP proxy service [Piracy]
===

Once you have the webclient and webserver, you can combine them to make a proxy service. Proxy services are great for piracy because then you are able to intercept the very HTTP traffic and then do analyse on them to save specific parts to a file (ex: a video on the web loads m3u8, the proxy can capture that and save files if the links in the m3u8 go through the proxy. For encrypted video stream in the body of HTTP decrypted by the client, the HTTP proxy will be of no use unless you wish to document the encrypted stream on the file system. this does not mean the idea of proxy is dead, because you just need to proxy the decryption algorithm instead, which may use a different protocol and may be very hard depending if hardware is involved)

/etc/hosts
---
Using /etc/hosts can make the request go to 127.0.0.1 which the proxy can listen on. Windows use `%windir%\System32\drivers\etc\hosts`

**Example**

```
127.0.0.1 example.com
```

in /etc/hosts will make example.com domain go to 127.0.0.1


Sample proxy
---
```
proxy = tcp stream socket listening on port 80

client = socket from proxy accept
server = tcp stream socket
connect server to example.com on port 80

receive from client and send to server
receive from server and send to client
```

Using a thread pool, you can accept many client requests instead of one at a time. Also, make sure to check if HTTP/1.0 or HTTP/1.1 and the `Connection` header to see if the socket should close. HTTP/1.0 closes if no `Connection` header. HTTP/1.1 do keep-alive if no `Connection` header.

Extension
---

If you wanting to proxy multiple domains, add them to the /etc/hosts. Change the proxy so that it doesnt connect to the server until the client is done with HTTP headers. With http headers copy, you can figure out the server that you should connect to in `Host` header. send the copy and then continue receive from client and send to server.

Using openssl [Util]
===

Generate an RSA private key
---
`$ openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out rsa.key`

(sidenote: rsa2048 is default)

if you wanting RSA1024 or RSA4096, change the `rsa_keygen_bits:2048` to `rsa_keygen_bits:1024` or `rsa_keygen_bits:4096`

OR use

`$ openssl genrsa -out rsa.key 2048`

replace 2048 with number of bit for the key

To encrypt the private key file with a password make a file called `password` in the directory and edit it so that it is the password on one line and ends with new line

`$ openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out rsa.key -aes256 -pass file:password`

OR use

`$ openssl genrsa -out rsa.key -aes256 -passout file:password 2048`

Replace aes256 with the cipher wanted to encrypt the key with

---

to view the key

`$ openssl pkey -in rsa.key -text -noout`

or the public part only

`$ openssl pkey -in rsa.key -text_pub -noout`

To view the key with password

`$ openssl pkey -in rsa.key -passin file:password -text -noout`

or the public part only

`$ openssl pkey -in rsa.key -passin file:password -text_pub -noout`


Generate an X509 root certificate
---
all root certs are self signed

First you need a certificate request. use private key for `-key`

`$ openssl req -new -key rsa.key -out cert.req`

to view

`$ openssl req -in cert.req -text -noout`

Then using the cert request, do

`$ openssl x509 -req -in cert.req -signkey rsa.key -out root.cert`

then, to view the cert, use

`$ openssl x509 -in root.cert -text -noout`


Generate a X509v3 root certificate
---
To do this, you need to specify the extensions in the cnf file. You can copy the authority key id from the signer certificate. the serial number of X509 certificate is arbitrary and chosen by the certificate authority.

A better way to generate the X509 root certificate

`$ openssl req -new -x509 -key rsa.key -out root.cert`

will generate a X509 root certificate with x509v3 extensions but cnf file is needed to control the extensions

**sample req config**

```
[req]
default_bits=2048
default_keyfile=default.key
distinguished_name=dn
attributes=req_attributes
prompt=no
req_extensions=v3

[req_attributes]

[dn]
C=NZ
ST=State
L=Locality
O=Organization
OU=Organizational Unit
CN=Common Name
[email protected]

[v3]
subjectKeyIdentifier=hash
basicConstraints=critical,CA:true
```

C is the 2 letters Country Code (ex: NZ), default.key file do not need to exist. only country name and common name are required in the dn section. Others like state, locality, etc are optionals

The subject key identifier can be set using a hex string if wanted

`subjectKeyIdentifier="111122223333444455556666777788889999AAAA"`

generate the certificate request

`$ openssl req -new -key rsa.key -config config.cnf -out cert.req`

**Sample ca config**

```
[ca]
default_ca=CA_default

[CA_default]
dir=.
database=$dir/index.txt
new_certs_dir=$dir/newcerts
certificate=$dir/ca.cert
serial=$dir/serial
private_key=$dir/ca.key
default_startdate=YYMMDDHHMMSSZ
default_enddate=YYMMDDHHMMSSZ
default_md=sha256
policy=policy_any
x509_extensions=v3

[policy_any]
countryName=supplied
stateOrProvinceName=optional
organizationName=optional
organizationalUnitName=optional
commonName=supplied
emailAddress=optional

[v3]
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid
basicConstraints=critical,CA:true
```

Create a empty file called index.txt and make a directory called newcerts. ca.cert and ca.key should be the certificate and private key for the ca. ca.key can be rsa.key for this because of self sign. Create the serial file and put in "02" or a hex like "111122223333444455556666777788889999AAAA" or add `rand_serial=yes` after the `serial=$dir/serial` line in cnf for a random serial which will not use serial file. but the serial file must exist even if not used by `rand_serial=yes`. default_md can be anything from the `$ openssl dgst -list` Change the default_startdate and default_enddate (ex: 210731123456Z is 2021-07-31 12:34:56 GMT)

then make the cert

`$ openssl ca -selfsign -key rsa.key -in cert.req -config config.cnf -out root.cert`

X509v3 Extensions
---

Edit the v3 section

**Sample Basic Constraints**

```
basicConstraints=CA:true
basicConstraints=CA:false
basicConstraints=critical,CA:true,pathlen:0
```

pathlen is the number of intermediate certificate that can be in the chain after this certificate. If it is 0 the certificate can only sign the server certificate

**Sample Key Usage**

can be any comma separated values from digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment, keyAgreement, keyCertSign, cRLSign, encipherOnly, decipherOnly. Can have critical

```
keyUsage=digitalSignature
keyUsage=critical,dataEncipherment,cRLSign
```

**Sample Extended Key Usage**

can be any comma separated values from serverAuth, clientAuth, codeSigning, emailProtection, timeStamping, OCSPSigning, ipsecIKE, msCodeInd, msCodeCom, msCTLSign, msEFS. Can have critical

```
extendedKeyUsage=emailProtection
extendedKeyUsage=serverAuth,clientAuth
extendedKeyUsage=critical,serverAuth,clientAuth
```

**Sample Certifiate Policies**

can be any comma separated values of policy oid. Can have policy sections but not shown here

```
certificatePolicies=1.2.3.4
certificatePolicies=1.2.3.4,1.1.1.1.1
```

**Sample Alternate DNS**

```
subjectAltName=@alt_names

[alt_names]
DNS.1=127.0.0.1
DNS.2=example.com
DNS.3=*.example.com
```

Generate a X509 certificate chain
---
In order to verify a certificate, the hash of the certificate is taken and signed by the signer cert's private key. using the same hash algorithm, and the public key of the signer cert, it checks if the signature matches (same hash).

if previous step from the guide was done, rename `rsa.key` to `root.key`. If not, this step needs a file called `root.cert` and `root.key` which is the root self sign certificate and the root private key

Generate private keys called intermediate.key and server.key and make requests for them. Make distinguished_name section something else for each request. for server.req use common name for domain name or use alternate dns extension. To create intermediate.cert, change the `certificate=$dir/ca.cert` to `certificate=$dir/root.cert` and change `private_key=$dir/ca.key` to `private_key=$dir/root.key`. change the dates and default_md and extensions if desire then

`$ openssl ca -in intermediate.req -config config.cnf -out intermediate.cert`

Then change `certificate=$dir/root.cert` to `certificate=$dir/intermediate.cert` and change `private_key=$dir/root.key` to `private_key=$dir/intermediate.key`. change the dates and default_md and extensions if desire then

`$ openssl ca -in server.req -config config.cnf -out server.cert`

Congratulations you now have a certificate chain!

Additional links
---
genpkey - generate a private key: https://man.archlinux.org/man/genpkey.1ssl

pkey - public or private key processing tool: https://man.archlinux.org/man/pkey.1ssl

config - OpenSSL CONF library configuration files: https://man.archlinux.org/man/config.5ssl.en

req - PKCS#10 certificate request and certificate generating utility: https://man.archlinux.org/man/req.1ssl

x509 - Certificate display and signing utility: https://man.archlinux.org/man/x509.1ssl

x509v3_config - X509 V3 certificate extension configuration format: https://man.archlinux.org/man/core/openssl/x509v3_config.5ssl.en

ca - sample minimal CA application: https://man.archlinux.org/man/ca.1ssl

Information missing here
---
Other Certificate Policy

Other Subject Alternative Names

CRL

CTL

SCT

OCSP

Related
---
LibreSSL

GnuPG

GnuTLS

HTTPS proxy service [Piracy]
===
Using information from openssl to create certificate, you only need to change sockets to ssl wrapped sockets which have a context of your certificate chain and server private key. This is really easy if you already have your code for your own HTTP proxy service. The ssl contexts will do the encryption. Remember to install your proxy's root certificate in both the certificate store of the ssl context and the client.

Sample proxy
---
```
context = new ssl context
give this ssl context your server certificate private key
give this ssl context your server certificate chain or
the root certificate if there is no chain and the common
name is domain

defaultContext = default ssl context

proxy = tcp stream socket
wrap proxy socket in the ssl context then
make proxy listen on port 443

client = socket from proxy accept
server = tcp stream socket
wrap server in the default ssl context then
connect server to example.com on port 443

receive from client and send to server
receive from server and send to client
```

Same advice and extension as the HTTP proxy, but just use ssl contexts. To proxy multiple domains, it may be needed to generate many contexts and certificates or use alternate DNS name with a single certificate.


HTTP/2 [Util] [Extension] (TODO)
===
HTTP/2 is released on 2015. It is the same fundamental design as HTTP/1.1 (use tcp/ip and tls, but for HTTP/2 tls is like a requirement and not optional). HTTP/2 no longer uses request pipeline as HTTP/1.1 (but replaces with multiplexing). HTTP/2 marks `Transfer-Encoding: chunked` HTTP header invalid. To upgrade to HTTP/2 a client sends header with `Connection: upgrade` and `Upgrade: HTTP/2.0` (todo: explain HTTP/2 frames)

HTTP/3 [Util] [Extension] (TODO)
===
HTTP/3 is released on 2019. HTTP/3 uses HTTP/2 api, but over udp instead of tcp. They made new protocol called QUIC which pushes packet resending and integrity information to application level instead of kernel level. This eliminates head of line blocking in HTTP/2 (todo: explain QUIC)

WebSockets (TODO)
===
The client will send a HTTP request with `Connection: upgrade` and `Upgrade: websocket`
(todo: examine the protocol and see if it can be used to intercept media)

Insecure WebSockets (TODO)
---
(todo: examine the protcol)

Secure WebSockets (TODO)
---

secure websockets operate on TLS (todo: examine the protcol)

Additional links
---
The WebSocket Protocol: https://datatracker.ietf.org/doc/html/rfc6455

Examining DirectShow WindowsAPI [Privacy] (TODO)
===
(todo: examine if it is possible to create a fake webcam feed using C++ and DirectShow WinAPI for privacy reasons)

Examining the ZIP File format [Steganography] [Privacy] (TODO)
===
The zip file format has an extra field in local file headers that may be used for metadata. the zip file format version made by can fingerprint (ex: 0x1e03 in the version made by is a little endian 2 byte integer specifying that the zip file was created on \*nix and the version support 3.0 of the zip standard). the external file attributes may be used for fingerprinting.

Because the central directory header of a zip file is at the end, it easy to append a zip file to like an image file (very basic steganography).
(todo: how to change file creation time using a zip file) (todo: more details)

Additional links
---
.ZIP File Format Specification 6.3.3: https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT

Examining the PNG File format [Steganography] [Privacy]
===
The png starts with hex 89504E470D0A1A0A and then png chunks come after. integers are big endian. Each chunk contains the 4 byte unsigned integer for the length of the data, the chunk type is the 4 byte ascii for the chunk type, the data, and then the CRC check for the data includes chunk type and chunk data but not the length. Each 4 byte ascii can be checked with bitwise and of 32 (&32) to make additional info about headers.

Because of ascii, if it lowercase, then the bitwise and is 32 and 0 for uppercase. if the first letter is uppercase, then the chunk is required in the image. it is lowercase then the chunk is not. If the second letter is uppercase, then the chunk type is public. If it is lowercase then the chunk type is private. the third letter case is reserved. if the fourth letter is uppercase, then the chunk should not be copy and if lowercase the chunk can be copy.

The CRC is done using the number EDB88320 for table calculation (read this hexadecimal number in binary from right to left to get the polynomial but without 32 power)

IHDR must be first chunk and IEND must be the last chunk. for the simplest image, one or more IDAT come after IHDR and those are the only 3 chunk type in the image. all IDAT are compressed using deflate

IHDR has a length of 13 and add chunk type ascii for hex 0000000D49484452. The first 4 byte are the image width, the next 4 bytes is the image height. The 5 next are all 1 byte which are the bit depth, the color type, the compression method, the filter method, and interlace method.

The example image is using bit depth 8, color type 6 (see below), compression method 0 (deflate), filter method 0 (null), interlace method 0 (null).

There are only 5 type of png image

**Truecolor and alpha = 6**

image has RGB values and alpha value

**Grayscale and alpha = 4**

image has gray value and alpha value

**Truecolor = 2**

image has RGB values

**Grayscale = 0**

image has gray value

**Indexed = 3**

image use a table to index at most 256 distinct colors. this table must be the PLTE chunk and only appear before any IDAT

Every pixel is made from channels with a sample depth. In the png, the channels have the same sample depth. The pixels come from samples of the channels or the index if indexed color. The bit depth represents the number of bits per sample or the bits in every table color if indexed color.

the image can use Adam7 interlace which make different images from the original pixels for each group of pixel in a pattern, each scanline is made from the pattern and represent that pixel group. Then all scanlines from top to bottom are turned into bytes, with the left pixel first and the right pixel last. it can use filter before compression to help the deflate compress. Then it is compressed stream and placed into image data chunks.

after the IHDR CRC comes the IDAT chunk hex ????????49444154. replace ? with length of chunk in hex big endian. Some programs limit IDAT length. Inside the IDAT chunk is deflate compressed data. The first byte is the compression method which is 08 (deflate), the next byte is the flags which is 1D (00011101 in binary). The first two bits 00 indicates compression with the fastest algorithm. the next bit 0 indicates no dictionary. The next bits 11101 are the check value for the two bytes where the first byte \* 256 + the entire flags byte must be multiple of 31.

8\*256 + 29 = 2048 + 29 = 2077 which is multiple of 31

then the compressed data is next. The byte will be the deflate method header and it 01 (binary 00000001). because last bit is 1, this is the last compressed block. The two bits to left of the 1 are the compression type which is 00 for no compression. other compression types are 01 for fixed Huffman, 10 for dynamic Huffman. 11 is error. Because of compression type no compression, next two byte is the length. This is the first byte + 256 \* second byte. then next two byte is the one complement of the length.

After this is the actual compressed image data (but this using type 00 for no compression) which mean only length from deflate header bytes after that. Then is the ADLER-32 of the uncompressed data. Then is the CRC of the image chunk.

How compute the ADLER-32 can be found in the RFC1950 and CRC32 can be found in the W3 specification

After this a 0 length IEND header 0000000049454E44AE426082 ends the image.

Steganography
---

1. The lower bit of selection of pixel sample can be change to make a binary message in the file which wont be visible to a human eye because it only affect lower bit. but only do this to image that is colorful

2. image is truecolor with alpha (type 6) and since the bit depth is 8, it fits ascii in the RGBA pattern. If G and A are 0, it is like wide string where R and B are hold the ascii. Because A are 0 in every pixel, the image look transparent but actual contains some data

download the image as image.png and try using `strings` in linux or open with notepad on windows

`$ strings image.png`

`$ strings -e l image.png`

notice the different result between the outputs.

3. Based on image chunks and not pixels there are still ways to do steganography. Example: send a lot of png, but make the number of IDAT chunks mean something. or use the PLTE chunk and other metadata chunks since it can has lots of bytes. Or make the specific chunk ordering different. there are a lot of way to do this based on only chunk alone

4. Or use pixels and chunks both for the steganography

Fingerprinting
---

It may be possible but very unlikely (i think) to discover the program used to make the picture by checking chunk ordering, chunk present and chunk count (ex: some program may include sRGB and gAMA chunk while other does not), private chunks, how and when the compression is done, color palette, color type, or other chunk metadata

some chunk like a text chunk can have metadata or compressed metadata for a keyword. Even if program that made picture is not know, this text metadata content may fingerprint.

Additional links
---

Portable Network Graphics (PNG) Specification (Second Edition): https://www.w3.org/TR/PNG/

ZLIB Compressed Data Format Specification version 3.3: https://datatracker.ietf.org/doc/html/rfc1950

Adam7 algorithm: https://en.wikipedia.org/wiki/Adam7_algorithm

Information missing here
---

Interlaced image

Other chunks (cHRM, gAMA, iCCP, sBIT, sRGB, bKGD, hIST, tRNS, pHYs, sPLT, tIME, iTXt, tEXt, zTXt) and their purpose and order in file

Filtered scanline functions

Changing File times [Privacy] (TODO)
===
Windows (TODO)
---
The creation time can be changed using a zip file (see: Examining the ZIP File format (TODO)), but you need to use MSDOS time and date (both 16bit integer) (todo: what is MSDOS time and date)
Otherwise, you need the `SystemTimeToFileTime` and `SetFileTime` functions of the windows api

Linux
---
You can use `stat` to see the info for a file
```
$ stat somefile
  File: somefile
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 801h/2049d	Inode: 6160770     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    user)   Gid: ( 1000/    user)
Access: 2021-07-14 06:38:12.138649334 +0000
Modify: 2021-07-14 06:38:12.138649334 +0000
Change: 2021-07-14 06:38:12.138649334 +0000
 Birth: -
```
we are interested in the access, modify, and change times

You can use `touch` to set time info for a file. you can only have full control over access and modify. change times are updated by the kernel to the current time. `touch -a` to change access. `touch -m` to change modify.
```
$ touch -a -d "2000-08-08 12:34:56.1234123" somefile
$ touch -m -d "2008-04-09 11:22:33.4444123" somefile
$ stat somefile
  File: somefile
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 801h/2049d	Inode: 6160770     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    user)   Gid: ( 1000/    user)
Access: 2000-08-08 12:34:56.123412300 +0000
Modify: 2008-04-09 11:22:33.444412300 +0000
Change: 2021-07-14 06:49:42.294649334 +0000
 Birth: -
```
The resolution of time may not be as precise because of the used file system.

(i do not recommend) to change the change time, you need to either patch the kernel for your system to stop automatically setting the change time, or you need to open the entire disk at a very low level and find and change data of the file's inode (none of these options are recommend by me!)

DEFLATE [Util] (TODO)
===
Huffman codes are prefix codes which mean that it's a set with no element being a prefix of any other. This variable length coding can be done based on frequency, which means that the shorter lengths get assigned to characters that appear more often. there is also backreference and length pairs that can repeat strings up to 32K before in the buffer. The Huffman tree can be reconstructed from a set of integers that are the bit lengths and count of bit lengths in a lexicographic ordering.
(todo: describe the compression algorithm)

Additional links
---
DEFLATE Compressed Data Format Specification version 1.3: https://datatracker.ietf.org/doc/html/rfc1951

Make a Firefox Addon [Util]
===

A Firefox addon can useful to add javascript into websites or to intercept HTTP requests. Also, there are many other types of Firefox addons to make. for each of the addons in the guide, make a directory if seems useful.

Content Scripts
---
This addon will add javascript into websites.

**File:** manifest.json

```json
{"manifest_version":2,
"name":"the name",
"version":"1.0",
"description":"the description",
"icons":{"32":"image.png"},
"content_scripts":[{
    "matches":["https://example.com/"],
    "js":["script.js"]
}]}
```

**File:** image.png

32x32 png image

**File:** script.js

```js
console.log('this is script')
```

Now install temporary addon and then go to https://example.com and open console. Try changing matches array in manifest.json by adding new urls or using wildcard like `https://example.com/*` or `*://*/*`. then see what happens when you visit urls. Change the script.js file to any javascript.

Intercept HTTP
---
This addon will intercept HTTP.

**File:** manifest.json

```json
{"manifest_version":2,
"name":"the name",
"version":"1.0",
"description":"the description",
"icons":{"32":"image.png"},
"permissions":["*://*/*","webRequest","webRequestBlocking"],
"background":{"scripts":["script.js"]}}
```

(sidenote: you can both use `content_scripts` and `background` in manifest.json)

**File:** image.png

32x32 png image

**File:** script.js

```js
function beforeRequest(detail){
if(!detail.url.indexOf('https://example.net')){
let f=browser.webRequest.filterResponseData(detail.requestId),
t=new TextEncoder();f.onstart=function(e){
f.write(t.encode('overwritten'));
f.close();f.disconnect()};
}
if(!detail.url.indexOf('https://example.com'))return{cancel:true};
return{};
}
browser.webRequest.onBeforeRequest.addListener(
beforeRequest,{urls:['*://*/*']},['blocking']);
```

Install temporary addon then this script will cancel all requests starting with https://example.com which can see if open console then go to network tab. Also will overwrite all requests start with https://example.net with text 'overwritten', try to open a new tab and go to https://example.com and then try https://example.net to see

Install Temporary Addon
---
Visit about:debugging then go to This Firefox link. Then load the addon by selecting any file (ex: manifest.json) in the addon directory. if using private mode, remember to visit about:addons to enable it for private mode

Additional links
---
Browser Extensions - Mozilla | MDN: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions

webRequest.onBeforeRequest - Mozilla | MDN: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/onBeforeRequest

webRequest.StreamFilter - Mozilla | MDN: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/StreamFilter

Crack your First Software [RE] (TODO)
===
some tools: IDA, radare2 (linux), gdb (linux), Hopper, ghidra, ollydbg (windows, obsolete), x64dbg, windbg
(todo: find software on web that would be easy for a beginner. go through thoughts and explain what to do to crack the software) (note: this is for beginner, do not discuss crack DRM here)

Cracking Java Software [RE] (TODO)
===

some tools: jdgui, java decompiler
(todo: find software on web that would be easy for beginning)

Cracking .NET Software [RE] (TODO)
===

some tools: ILSpy (see source), de4dot (deobfuscate)
(todo: get software on web that would be easy for beginner)

Making your First keygen [RE] (TODO)
===

(todo: find easy software to make a keygen for, then describe thought process and how it is can be done)

Cracking DRM [RE] (TODO)
===

some tools: DeDRM

handbrake with some libraries can also do it

Always online DRM can be bypassed if a server is programmed for it. /etc/hosts will be useful like in HTTP or HTTPS proxy

(todo: more research and aggregate and post back here)