OpenLDAP - How to enable CRL check for LDAP TLS connections?

Refer the old post to understand more about certificate chain verification and CRL.

Here we will see how to establish a secure connection using OpenLDAP. OpenLDAP provides set of "set" options through which we can enable the CRL check, supply required certificates and we can set the verify call back. Using this verify call back we can control OpenLDAP behavior on each certificate verification.

Below example is a typical client process which is providing CA certificate and during TLS connection server will be sending the EE (along with intermediate certificates) to form a complete chain. During the connection negotiation, these certificates are validated.

Once the CRL check is enabled, during certificate verification, OpenSSL calls default call back which has the default implementation of breaking the verification once the error occurred.

Here in the below example, we are registering LDAP call back, using this we will get access to SSL store objects and we can set the SSL call back. In the SSL verify call back we will ignore some set of errors like  X509_V_ERR_UNABLE_TO_GET_CRL", "X509_V_ERR_CRL_HAS_EXPIRED", "X509_V_ERR_CRL_NOT_YET_VALID"  and proceed with the connection.

Read More

OpenSSL - How to verify certificate chain with CRL?

When any certificate is issued, it has a validity period which is defined by the Certification Authority. Usually, this is one or two years.

However, sometimes certificates should not be honored even during their validity period. For example, if the private key associated with a certificate is lost or exposed, then any authentication using that certificate should be denied.

That's where CRL comes into the picture. A CRL is a Certificate Revocation List which contains the list of certificates revoked by the authority.
These CRLs are usually stored in a centralized locations called CRL Distribution Point. This distribution point URI/URL will be made available in the certificate extensions by the authority.

Now let's say we have certificate chain like rca->ica->ee and CRL issued by rca and ica, How can we verify the certificate chain?

Command line:

openssl verify -crl_check -verbose -CAfile <(cat rca.pem ica.pem crl_rca.pem crl_ica.pem) ee.pem

C++ way:

Here is the sample class called CertificateStore which is used to verify the certificate chain with CRL.
This class creates a global store and a store context (ctx). All the required flags and the directory paths are set to the global store and certificate chain verify happens through store ctx.

This store ctx can be used only once to verify the certificate chain. If you want to verify new chain, new store ctx has to be created but this new store ctx can be initialized from the global store using which it can inherit the properties of the global store.

We can add certificates and CRLs to the store individually using X509_STORE_add_cer/X509_STORE_add_crl methods or we can use the directory lookup using the X509_LOOKUP_add_dir method.

If you are using hash directory lookup, OpenSSL computes the hash of the issuer and searches for the file with the name which matches <hash>.rN.
More details in https://www.openssl.org/docs/man1.1.0/crypto/X509_LOOKUP_file.html

Read More

Python - Selecting a random item from a list or tuple or dict

Say you have a list with 'n' number of items and you wish to get some random item from that list, How do you do it ?

Python has inbuilt 'random' package to do this.

To get one random item from the list:

>>> import random
>>> items = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
>>> print random.choice(items)
'c'

There’s an equally simple way to select n items from the sequence:

>>> import random
>>> items = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
>>> print random.sample(items, 2)
['e', 'a']

Read More

ldapmodify failed with error "dn: attribute type undefined"

I was facing this strange error "dn: attribute type undefined" while I am trying to delete attributes from the LDAP. My ldif file has two attributes to be deleted.

cat /tmp/modifyattr.ldif

dn: fsFgId=Child2, fsFgId=Child1, fsClsId=Root
changetype: modify
delete: fsListeningHost

dn: fsFgId=Child2, fsFgId=Child1, fsClsId=Root
changetype: modify
delete: fsPort

When ldapmodify command is executed with the above ldif file it says "Undefined attribute type".

modifying entry "fsFgId=Child2, fsFgId=Child1, fsClsId=Root"
modify complete
ldapmodify: Undefined attribute type (17)
additional info: dn: attribute type undefined

I googled for this error, every where I found suggestion to add extra blank line before dn. It didn't work for me. After a lot of research and experiments I found that I was adding extra space along with the blank line !!! Because I was creating this file using bash script, I didnt notice that extra space.

Finally it worked like charm.

Read More

Secure way of deleting files in Unix

As I mentioned in the previous post there are simple techniques that can recover your deleted file (by using simple 'grep' or 'strings' utilities).
Even some data recovery tools does the same thing. So if you want to delete some data on the disk without being worried about the retrieval, then
you should probably over write the disk which has your file content.

shred utility available in Linux does the same thing.
shred actually overwrite the specified file repeatedly, in order to make it harder for even very expensive hardware probing to recover the data.
By default shred overwrites the file 25 times with the junk data.

you can chose to remove the file after over writing using -u (unlink)
There are multiple options with shred to explore.

$ shred -u filetobedeleted.txt

Just to see how it works, let say my script is writing some data to the file 'testlog.log' repeatedly after every 1 min.
I am tailing the file in one terminal. And in other terminal I did execute shred.
 

$ sh writetodisk.sh &

$ tail -f testlog.log
aaa
bbb
ccc
ddd

$ shred testlog.log

Now observe the terminal one

$ tail -f testlog.log
aaa
bbb
ccc
ddd
\{XÁÀà_ç æIƒòDÊ5žq­Æ 8<TÝõ ¬ S õŸt\Ê1’ïNÐ , éM‚?$Väé@. l"®ÎþÌÕæ ‡Ù+Ž’ bªO"× #f©ÎçN‰/h÷¡çÊhÇöŸz!*ÀA?RAo%æ} ÛZ½PSàpû7Íû3U_ ’e^u÷züê¾Ú6󚶄Ë[Fœ;½êê±î÷
                                                           Cý ]¤¥ˆi                                                                                  ÕÎ8ƒ:SÎq3®B h€'Q“ãªF¹X‘Q'†GÁ–oõ»hï eþ:½U4Úy_£È‘”f}"J_ŠÒ‡±Ê0íÕwº }rºŸoÇpÜ Wá‚À°xfeÒ?ÕC·         ‰JðhJë ™ÀQêM]ÞÑÅ,A {9b ÑùÇ@©}ÅŠ½°Ò¡øÜK-òõ ªLoLƒü
GýÑeÈ#WsG`Þ¼µÅ"–> T/~ [ºÝ ¸ýŒ<C8îzD±š¨š J
#Lwk{lû´köAٍ^0ê(9¿Ó Xnš¼¼ýc+7×Ãó ‡@ ;¥
                                        ŽBýˆÔ
                                              ÀF ⍠?’‰´q’+iQ‰ Y¸¯`± {·;²&%6ÈÄLYdù½­ š¼ÑÖi…ö±É* ÝÜ(Y2Ðc FÔ]þŠ ˜° ˜ƒTãðõ,l‚šl„bÜ8Å òU='µ YR™&iõqmôT ¤¿)“G[¡9îÎD $ˆÉšDÒ–„xFÀjKNs„)½3̆^¹°w

you can see that the shred filled the contents of the file with garbage data.

Read More

How to recover a file that was removed using 'rm' command ?

In Unix like file systems, the system uses 'hard links' to point to piece of data that you write to the disk.
So when you create a file, you also create its first hard link. You can create multiple hard links using 'ln' command.
When you "delete" a file using rm command, normally you are only deleting the hard link.

If all hard links to a particular file are deleted, then the system removes only the reference to the data and indicate that the blocks as free. But it won't actually delete the file.

If your deleted file is opened by any running process then it means you still have one link left to your file !!.
check if any process who works on your file using lsof command

$ lsof | grep "myfile.txt"
COMMAND    PID     USER   FD      TYPE    DEVICE   SIZE     NODE    NAME
pgm-name   7099    root   25r    REG     254,0    349      16080   /tmp/myfile.txt

Using the process and file descriptor you can try copying the file

$ cp /proc/7099/fd/25 /mydir/restore.txt

If lsof didn't list your file then you could try to locate your data reading directly from the device.
But this works only if blocks containing your files haven't been claimed for something else.

To make sure no one else over writes that free blocks, immediately remount the file system with read-only and then search for your file.

$ mount -o ro,remount /dev/sda1
$ grep -a -C100 "unique string" /dev/sda1 > file.txt

Replace /dev/sda1 with the device that the file was on and replace 'string' with the unique string in your file.
This does is it searches for the string on the device and then returns 100 lines of context and puts it in file.txt.
If you need more lines returned just adjust -C options as appropriate. Alternatively you can use -A, -B options with grep to print lines before and after the matched string.

You might get a bunch of extra garbage date, mostly some binary data but you can get your data back.
If you don't want this binary data then you can apply 'string' on the device and grep for the unique string

$ strings /dev/sda1 | grep -C100 "unique string" > file.txt

Read More

Calculating time differences in python

Recently I had a situation where I need to calculate the timer efficiency.
I have a C++ timer that calls my function after every 5 sec. My function does some critical operation and logs one statement to syslog.
When I observed the logs I found that there is delay in my function execution, slowly the timer started drifting in result and the next function calls are getting delayed !!

So I wanted to calculate how many times the timer has delayed in a day. I grepped for the particular log in my syslog and redirected to a file.

file format is like this:
Jan 29 06:34:24
Jan 29 06:34:29
Jan 29 06:34:34
Jan 29 06:34:39
..
..
Now I should compare the lines in the file and log if the time difference is greater than 5 sec.
compare line 1 with line 2
line 2 with line 3, then line 3 with line 4 and so on...



Here the bad thing is f.readlines() ... It will load whole file in to list and tried to read 2 lines at a time.
If anybody reads this post :P and if you know any better working solution please share. :)

Read More