GL.IB.LY » grep

Ann skips bail. Cue forensics puzzle.

tariq — Fri, 27 Nov 2009 09:00:01 +0000

Found a website and a forensics contest yesterday quite by accident. I was waiting for somebody before going out for the night and I thought this might be a little fun while I waited. Now the contest had closed and the results where available, which I ignored until the end and went straight to Puzzle #2: Ann skips bail.

The puzzle revolves around a packet capture of Ann’s network taken by wily investigators before she skipped bail. Police are confident that she communicated with a secret lover prior to her disappearance. And so follows a number of competition questions. It is important to note that the organizers are looking for the most elegant solutions, and you won’t see that here. What you will see is how to solve the puzzle very quickly.

1. What is Ann’s email address?
2. What is Ann’s email password?
3. What is Ann’s secret lover’s email address?
4. What two items did Ann tell her secret lover to bring?
5. What is the NAME of the attachment Ann sent to her secret lover?
6. What is the MD5sum of the attachment Ann sent to her secret lover?
7. In what CITY and COUNTRY is their rendez-vous point?
8. What is the MD5sum of the image embedded in the document?

I downloaded the packet dump file from the organiser’s site and verified the hash using md5 as I am on a Mac, otherwise md5sum does the job.

I fired up wireshark and opened the packet capture file. There appeared to be a good bit of SMTP traffic. So I did a quick

grep -an "To:.*\|From:*\|Subject:.*" evidence02.pcap

on the packet dump which revealed the following.

From: "Ann Dercover" To: Subject: lunch next week From: "Ann Dercover" To: Subject: rendezvous

Its clear that Ann Dercover’s email address is sneakyg33k@aol.com. This is the answer to question 1. We also see she sent two emails. One to mistersecretx@aol.com, could this be Ann’s secret lover? I expanded the grep to

grep -aA50 "mistersecretx@aol.com" evidence02.pcap

This gives me 50 lines after and including lines matching mistersecretx@aol.com.

To: Subject: rendezvous Date: Sat, 10 Oct 2009 07:38:10 -0600 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_000D_01CA497C.9DEC1E70" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180

This is a multi-part message in MIME format.

——=_NextPart_000_000D_01CA497C.9DEC1E70
Content-Type: multipart/alternative;
boundary=”—-=_NextPart_001_000E_01CA497C.9DEC1E70″

——=_NextPart_001_000E_01CA497C.9DEC1E70
Content-Type: text/plain;
charset=”iso-8859-1″
Content-Transfer-Encoding: quoted-printable

Hi sweetheart! Bring your fake passport and a bathing suit. Address =
attached. love, Ann
——=_NextPart_001_000E_01CA497C.9DEC1E70
Content-Type: text/html;
charset=”iso-8859-1″
Content-Transfer-Encoding: quoted-printable

…SNIP…

Hi sweetheart! Bring your fake passport =
and a=20
bathing suit. Address attached. love, Ann

…SNIP…

——=_NextPart_000_000D_01CA497C.9DEC1E70
Content-Type: application/octet-stream;
name=”secretrendezvous.docx”
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename=”secretrendezvous.docx”

Wow. So mistersecretx@aol.com IS the email address of Ann’s lover. This is the answer to question 3. We also see in her message she instructs them to “bring your fake passport and bathing suit”, this is the answer to question 4. Towards the bottom we see an attachment that probably appears later than the lines we grep’d called secretrendezvous.docx which will appear base64 encoding. This is the answer to question 5.

I then returned to wireshark, looking down through the packets I quickly see SMTP traffic with C: DATA fragment which tells us this traffic was broken up into smaller pieces. This is likely to be an email with a large attachment. I right clicked on one of these packets as shown below and clicked on Follow TCP Stream as shown below.

This gives me the following.

220 cia-mc07.mx.aol.com ESMTP mail_cia-mc07.1; Sat, 10 Oct 2009 15:37:56 -0400 EHLO annlaptop 250-cia-mc07.mx.aol.com host-69-140-19-190.static.comcast.net 250-AUTH=LOGIN PLAIN XAOL-UAS-MB 250-AUTH LOGIN PLAIN XAOL-UAS-MB 250-STARTTLS 250-CHUNKING 250-BINARYMIME 250-X-AOL-FWD-BY-REF 250-X-AOL-DIV_TAG 250-X-AOL-OUTBOX-COPY 250 HELP AUTH LOGIN 334 VXNlcm5hbWU6 c25lYWt5ZzMza0Bhb2wuY29t 334 UGFzc3dvcmQ6 NTU4cjAwbHo= 235 AUTHENTICATION SUCCESSFUL MAIL FROM: 250 OK RCPT TO: 250 OK DATA 354 START MAIL INPUT, END WITH "." ON A LINE BY ITSELF Message-ID: <001101ca49ae$e93e45b0$9f01a8c0@annlaptop> From: "Ann Dercover" To: Subject: rendezvous Date: Sat, 10 Oct 2009 07:38:10 -0600 MIME-Version: 1.0 Content-Type: multipart/mixed; .boundary="----=_NextPart_000_000D_01CA497C.9DEC1E70" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2180 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180

This is a multi-part message in MIME format.

——=_NextPart_000_000D_01CA497C.9DEC1E70
Content-Type: multipart/alternative;
.boundary=”—-=_NextPart_001_000E_01CA497C.9DEC1E70″

——=_NextPart_001_000E_01CA497C.9DEC1E70
Content-Type: text/plain;
.charset=”iso-8859-1″
Content-Transfer-Encoding: quoted-printable

Hi sweetheart! Bring your fake passport and a bathing suit. Address =
attached. love, Ann
——=_NextPart_001_000E_01CA497C.9DEC1E70
Content-Type: text/html;
.charset=”iso-8859-1″
Content-Transfer-Encoding: quoted-printable

charset=3Diso-8859-1″>

Hi sweetheart! Bring your fake passport =
and a=20
bathing suit. Address attached. love, Ann

——=_NextPart_001_000E_01CA497C.9DEC1E70–

——=_NextPart_000_000D_01CA497C.9DEC1E70
Content-Type: application/octet-stream;
.name=”secretrendezvous.docx”
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
.filename=”secretrendezvous.docx”

UEsDBBQABgAIAAAAIQDleUAGfwEAANcFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAAC
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC0
VMluwjAQvVfqP0S+VsTQQ1VVBA5dji1S6QcYexKsepNttr/vOEBEKQSpwCVSPH7LPI/dHy61yubg
g7SmIL28SzIw3AppqoJ8jd86jyQLkRnBlDVQkBUEMhzc3vTHKwchQ7QJBZnG6J4oDXwKmoXcOjBY
Ka3XLOKvr6hj/JtVQO+73QfKrYlgYicmDjLov0DJZipmr0tcXjtxpiLZ83pfkiqI1Amf1ulBhAcV
9iDMOSU5i9gbnRux56uz8ZQjst4TptKFOzR+RCFVfnvaFdjgPjBMLwVkI+bjO9PonC6sF1RYPtPY
dd5Oc8CnLUvJocEnNucthxDwlLTKm4pm0mz9H/VhZnoCHpGXN9JQnzQR4kpBuLyDNW+bPIY18tYF
imd3tj6kgRUgOngeDnyU0MzP0fwDxIjpX6P5DXNb+/UoRrymQOtv7+wMapqTkiVe5TGbKDhb78/4
N9QnTSxg8nm19HfI24w088et/0cY2zcroQ9MHa2f5cEPAAAA//8DAFBLAwQUAAYACAAAACEAHpEa
t/MAAABOAgAACwAIAl9yZWxzLy5yZWxzIKIEAiigAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIyS20oDQQyG7wXfYch9N9sKItLZ3kihdyLr
A4SZ7AF3Dsyk2r69oyC6UNte5vTny0/Wm4Ob1DunPAavYVnVoNibYEffa3htt4sHUFnIW5qCZw1H
zrBpbm/WLzyRlKE8jDGrouKzhkEkPiJmM7CjXIXIvlS6kBxJCVOPkcwb9Yyrur7H9FcDmpmm2lkN
aWfvQLXHWDZf1g5dNxp+Cmbv2MuJFcgHYW/ZLmIqbEnGco1qKfUsGmwwzyWdkWKsCjbgaaLV9UT/
X4uOhSwJoQmJz/N8dZwDWl4PdNmiecevOx8hWSwWfXv7Q4OzL2g+AQAA//8DAFBLAwQUAAYACAAA
ACEApOAquCABAAA6BAAAHAAIAXdvcmQvX3JlbHMvZG9jdW1lbnQueG1sLnJlbHMgogQBKKAAAQAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACsk01OwzAQhfdI3MHynjgpUBCq0w1C6hbCAdxkkljE
P7KnQG7PKFKbVJSwycbSvCjvfZ7xbLbfpmOfEKJ2VvIsSTkDW7pK20by9+Ll5pGziMpWqnMWJO8h
8m1+fbV5hU4h/RRb7SMjFxslbxH9kxCxbMGomDgPlr7ULhiFVIZGeFV+qAbEKk3XIkw9eH7myXaV
5GFX3XJW9J6S//d2da1LeHblwYDFCxEiAiLdLJKnCg2g5EclIU4uLiM8LImA1BoY84dSDGc2x7Ba
kiFi39EcxyYM9Vx8tmS8PZg9BJrDSHCS5iDWS0LUzmKh9t1kFidpDuJ+SQhtaBfGLhiotBKDmCWe

…SNIP: not publishing the whole thing, its a bit long. See the whole thing?

JZ2ekPwNAAD//wMAUEsBAi0AFAAGAAgAAAAhAOV5QAZ/AQAA1wUAABMAAAAAAAAAAAAAAAAAAAAA
AFtDb250ZW50X1R5cGVzXS54bWxQSwECLQAUAAYACAAAACEAHpEat/MAAABOAgAACwAAAAAAAAAA
AAAAAAC4AwAAX3JlbHMvLnJlbHNQSwECLQAUAAYACAAAACEApOAquCABAAA6BAAAHAAAAAAAAAAA
AAAAAADcBgAAd29yZC9fcmVscy9kb2N1bWVudC54bWwucmVsc1BLAQItABQABgAIAAAAIQA6Q0kI
FQQAAFgKAAARAAAAAAAAAAAAAAAAAD4JAAB3b3JkL2RvY3VtZW50LnhtbFBLAQItAAoAAAAAAAAA
IQBg7VaATPYCAEz2AgAVAAAAAAAAAAAAAAAAAIINAAB3b3JkL21lZGlhL2ltYWdlMS5wbmdQSwEC
LQAUAAYACAAAACEAlrWt4pYGAABQGwAAFQAAAAAAAAAAAAAAAAABBAMAd29yZC90aGVtZS90aGVt
ZTEueG1sUEsBAi0AFAAGAAgAAAAhAIkUT0qVAwAAcQgAABEAAAAAAAAAAAAAAAAAygoDAHdvcmQv
c2V0dGluZ3MueG1sUEsBAi0AFAAGAAgAAAAhAErYipK7AAAABAEAABQAAAAAAAAAAAAAAAAAjg4D
AHdvcmQvd2ViU2V0dGluZ3MueG1sUEsBAi0AFAAGAAgAAAAhADVKHsm+CgAALFoAAA8AAAAAAAAA
AAAAAAAAew8DAHdvcmQvc3R5bGVzLnhtbFBLAQItABQABgAIAAAAIQCQUuGobwEAANcCAAARAAAA
AAAAAAAAAAAAAGYaAwBkb2NQcm9wcy9jb3JlLnhtbFBLAQItABQABgAIAAAAIQAXVdHWCQQAAMsZ
AAASAAAAAAAAAAAAAAAAAAwdAwB3b3JkL251bWJlcmluZy54bWxQSwECLQAUAAYACAAAACEAu6G5
NXECAACGCAAAEgAAAAAAAAAAAAAAAABFIQMAd29yZC9mb250VGFibGUueG1sUEsBAi0AFAAGAAgA
AAAhAKVR8wbYAQAA2QMAABAAAAAAAAAAAAAAAAAA5iMDAGRvY1Byb3BzL2FwcC54bWxQSwUGAAAA
AA0ADQBEAwAA9CYDAAAA

——=_NextPart_000_000D_01CA497C.9DEC1E70–

.
250 OK
QUIT
221 SERVICE CLOSING CHANNEL

You may or may not realise that parts of the communication are base64 encoded. Lets take a look at some information encoded at the beginning of this communication again.

AUTH LOGIN 334 VXNlcm5hbWU6 c25lYWt5ZzMza0Bhb2wuY29t 334 UGFzc3dvcmQ6 NTU4cjAwbHo= 235 AUTHENTICATION SUCCESSFUL

Here Ann is authenticating with the service. Her responses are shown in red, and as you can see they’re a bit cryptic; however, they look like they are encoded in base64. So we run the following two commands.

$ echo "c25lYWt5ZzMza0Bhb2wuY29t" | openssl base64 -d sneakyg33k@aol.com $ echo "NTU4cjAwbHo=" | openssl base64 -d 558r00lz

Note: $ is the command prompt, what follows it is the command with output in green.

So we find Ann’s email password is 558r00lz. This the answer to question 2.

Next we have a look at the attachment which is base64 encoded. We copy all the blue text above and paste into a file called attachment.b64 and issue the following commands.

$ openssl base64 -d < attachment.b64 > secretrendezvous.docx $ md5 secretrendezvous.docx MD5 (secretrendezvous.docx) = 9e423e11db88f01bbff81172839e1923

This decodes the data and outputs to secretrendezvous.docx. We can open the file, verifying it is good and thus the md5 sum of 9e423e11db88f01bbff81172839e1923 is the answer to question 6. When we open the file we see an image like the one below.

This tells us that Ann was off to Playa del Carmen in Mexico. This is the answer to question 7. We only now need to get the md5 sum of the image in the document. This is easy enough as we can just do the following:

$ unzip secretrendezvous.docx -d attachment Archive: out-1.docx inflating: attachment/[Content_Types].xml inflating: attachment/_rels/.rels inflating: attachment/word/_rels/document.xml.rels inflating: attachment/word/document.xml extracting: attachment/word/media/image1.png inflating: attachment/word/theme/theme1.xml inflating: attachment/word/settings.xml inflating: attachment/word/webSettings.xml inflating: attachment/word/styles.xml inflating: attachment/docProps/core.xml inflating: attachment/word/numbering.xml inflating: attachment/word/fontTable.xml inflating: attachment/docProps/app.xml

The only image file in there is attachment/word/media/image1.png. Open it up to verify it is the business and then just do

$ md5 attachment/word/media/image1.png MD5 (attachment/word/media/image1.png) = aadeace50997b1ba24b09ac2ef1940b7

This is the answer to question 8, and we’re finished. That was quick! The answers have been published so you can verify. Now that person has turned up and is dragging me away from my computer so that’s all for now.

The post Ann skips bail. Cue forensics puzzle. appeared first on GL.IB.LY.

All configuration variables in Moodle code – Part two

tariq — Wed, 24 Jun 2009 17:53:03 +0000

Earlier we looked at how you can extract a list of all the $CFG variables in your Moodle code. Now that’s not of much use! We need to know where in the code all these variables hide. I feel some spooky awk a coming on.

Please note this doesn’t show all the variables available in Moodle, it shows all the Moodle variables in a particular codebase. Below output is shown from one of my own codebases which is a little different to what you might expect to see in MOODLE_19_STABLEIn All configuration variables in Moodle code we came up with:

grep -r –no-filename -o ‘\$CFG->[a-z][a-z]*’ | sort -u

This gives us some useful information but hides a lot of the detail. Let’s try and include which files the variable appears in even the line numbers. For this we’ll go back to:

grep -r -no ‘\$CFG->[a-z][a-z]*’

This will print line numbers for matches. Hmmm, what happens if two or more matches are found on one line? Well the second var will be printed without the file and line number — silly badness, see below!

./admin/auth.php:11:$CFG->libdir
./admin/auth.php:12:$CFG->libdir
./admin/auth.php:17:$CFG->wwwroot
$CFG->admin
./admin/auth.php:26:$CFG->auth
./admin/auth.php:29:$CFG->auth
./admin/auth.php:52:$CFG->registerauth
./admin/auth_config.php:7:$CFG->libdir
./admin/auth_config.php:11:$CFG->pagepath

To add some context to $CFG->admin (which appears on line 17 of admin/auth.php) we introduce awk to the fray.

grep -r -no ‘\$CFG->[a-z][a-z]*’ . | awk -F: ‘BEGIN{ s=“” }{ if($3) { print $0; s=$1“:”$2“:”; }else print s$0; }‘

Here, awk saves the previous line’s first and second fields (separated by ‘:’) if the third field is not empty. If the third field is empty it prepends the first and second fields of the previous line to whatever is on the current line. Nifty, eh? So when we run our command line we now get:

./admin/auth.php:11:$CFG->libdir
./admin/auth.php:12:$CFG->libdir
./admin/auth.php:17:$CFG->wwwroot
./admin/auth.php:17:$CFG->admin
./admin/auth.php:26:$CFG->auth
./admin/auth.php:29:$CFG->auth
./admin/auth.php:52:$CFG->registerauth
./admin/auth_config.php:7:$CFG->libdir
./admin/auth_config.php:11:$CFG->pagepath

Much better! Now that we are getting all the data we can do something useful. Let’s sort everything for convenience.

grep -r -no ‘\$CFG->[a-z][a-z]*’ . | awk -F: ‘BEGIN{ s=“” }{ if($3) { print $0; s=$1“:”$2“:”; }else print s$0; }‘ | sort -t: -k3

We are sorting our data by the third field in each row. So we tell sort to use ‘:’ as our field delimiter with the -t and we tell sort to use the third field with -k3.

Now we’ll make everything prettier by using awk again (our data should be small enough). We use awk to keep track of the last file and variable awk sees and then make formatting improvements as these values change; e.g. if the Variable changes from $CFG->admin to $CFG->wwwroot then prepend variable name with \n\n which inserts two line returns. This all looks pretty scary…

grep -r -no ‘\$CFG->[a-z][a-z]*’ . | awk -F: ‘BEGIN{ s=“” }{ if($3) { print $0; s=$1“:”$2“:”; }else print s$0; }‘ | sort -t: -k3 | awk -F: ‘BEGIN{ v=“”; f=“”; }{ if(v!=$3) { v = $3; f=“”; printf “\n\n”$3; } if(f!=$1) { f = $1; printf “\n\t”$1; } printf “:”$2 }‘

This gives us something nice and pretty for our terminal window.

...
$CFG->apachemaxmem
        ./lib/moodlelib.php:7544:7545:7547:7548
$CFG->aspellextradicts
        ./lib/speller/server-scripts/spellchecker.php:27:28
$CFG->aspellpath
        ./admin/settings/appearance.php:57
        ./lib/adminlib.php:2870:2881
        ./lib/editor/htmlarea/htmlarea.php:174:174:279:279
        ./lib/javascript.php:8
        ./lib/speller/changes.txt:19:1:29
        ./lib/speller/server-scripts/spellchecker.php:14:5
        ./lib/weblib.php:4988
...

We can, however, spruce things up a bit for the web:

”$3“

”; } if(f!=$1) { f = $1; printf “
font-weight: bold\“>”$1“”; } printf “:”$2 }‘

This gives us something like the following output:

$CFG->ADMIN

./admin/auth.php:17
./admin/auth_config.php:18
./admin/block.php:44
./admin/blocks.php:157:172:237
./admin/collect/cron.php:11
./admin/collect/index.php:108:114:120:126:14:22:22:43:80:91
./admin/collect/lib.php:31:709:75:76
./admin/collect/queue.php:105:106:123:13
./admin/cron.php:168:170
./admin/environment.php:124
./admin/filter.php:16
./admin/filters.php:11
./admin/index.php:422:427:432:437:441:445:448:451:456:459:460:463:464:465:468:471:471:523:602
./admin/lang.php:102:122:127:130:131:132:142:493:754
./admin/langdoc.php:101
./admin/mnet/access_control.php:48:63:76:87
./admin/mnet/adminlib.php:75
./admin/mnet/delete.html:3:4
./admin/mnet/enr_courses.php:40
./admin/mnet/enr_hosts.php:40:52
./admin/module.php:41
./admin/modules.php:201
./admin/pagelib.php:72
./admin/register.php:37
./admin/report/assess/index.php:12:13:193:50
./admin/report/certify/index.php:11:159:270
./admin/report/courseoverview/index.php:18:88
./admin/report/courseoverview/reportsgraph.php:32
./admin/report/lastaccess/index.php:12
./admin/report/lateforleveltwo/index.php:14:86:90
./admin/report/numberofcourses/index.php:12
./admin/report/stats/index.php:31:47
./admin/roles/assign.php:381
./admin/roles/managetabs.php:10:12:14
./admin/roles/override.php:160:187
./admin/roles/tabs.php:182:190
./admin/settings.php:19:42:6
./admin/settings/collect.php:12:17:18:19
./admin/settings/courses.php:12
./admin/settings/frontpage.php:56
./admin/settings/language.php:24:25
./admin/settings/location.php:23
./admin/settings/misc.php:19:23:24:25:26
./admin/settings/mnet.php:13:16:19:22:9
./admin/settings/plugins.php:141:33:41:64
./admin/settings/server.php:161:218:219:247:248
./admin/settings/top.php:16:19:36:46:55
./admin/settings/unsupported.php:10:7:8:9
./admin/settings/users.php:58:71:72:74:75:81:82
./admin/stickyblocks.php:47:73
./admin/upgradesettings.php:31
./admin/uploaduser.php:65:66
./admin/user/user_bulk.php:22:23:24:25:26:5:6
./admin/user/user_bulk_confirm.php:14
./admin/user/user_bulk_delete.php:14
./admin/user/user_bulk_display.php:11
./admin/user/user_bulk_download.php:14
./admin/user/user_bulk_message.php:13
./admin/xmldb/actions/XMLDBAction.class.php:161
./admin/xmldb/index.php:106:78:88
./admin/xmldb/javascript.php:42:43:49:50
./backup/backup.php:104:76:91
./backup/backup_scheduled.php:180
./backup/log.php:11
./backup/restore.php:103:96
./backup/restore_check.html:268
./blocks/admin/block_admin.php:67
./blocks/admin_bookmarks/block_admin_bookmarks.php:55
./blocks/admin_bookmarks/create.php:36
./blocks/admin_bookmarks/delete.php:32
./blocks/admin_tree/block_admin_tree.php:212:67
./blocks/certify/certify.php:11
./blocks/certify/close.php:86
./blocks/certify/config_instance.html:13
./blocks/course_summary/block_course_summary.php:33
./blocks/moodleblock.class.php:475
./blocks/reports/block_reports.php:41:50:85
./blocks/trouble_ticket/thanks.php:25
./calendar/event.php:64
./calendar/export.php:19
./calendar/view.php:54
./config-dist.php:125
./config.php:23
./course/category.php:140:384
./course/delete.php:34:65
./course/edit.php:118:150
./course/editcategory.php:179:323
./course/importstudents.php:19
./course/mod.php:706
./course/modedit.php:446
./course/report/stats/index.php:28
./course/search.php:275
./enrol/authorize/enrol.php:414:746
./enrol/imsenterprise/importnow.php:13
./filter/algebra/algebradebug.php:261
./filter/tex/texdebug.php:263
./grade/report/grader/preferences.php:87
./index.php:38:56:62:74:83
./install.php:139:547
./lib/adminlib.php:3458:3498:3822:3972:3977:3985:4172:4175:4176:4335:776
./lib/db/mysql.php:1473
./lib/db/postgres7.php:1216
./lib/ddllib.php:818:820
./lib/dmllib.php:510
./lib/environmentlib.php:320:323
./lib/moodlelib.php:5099
./lib/pagelib.php:471
./lib/setup.php:184:185
./login/index.php:9
./mod/assignment/lib.php:2837
./mod/glossary/formats.php:27:39
./mod/hotpot/db/update_to_v2.php:519
./my/pagelib.php:59
./question/upgrade.php:90
./user/editadvanced.php:153:158
./user/index.php:100:542:548
./user/tabs.php:225:233:235

$CFG->ADMINBLOCKS

./admin/index.php:522

$CFG->ADMINEDITALWAYS

./config-dist.php:147
./search/documents/forum_document.php:169

$CFG->ADMINSEESALL

./calendar/lib.php:1426:1427

$CFG->ADMINUSEHTMLEDITOR

./admin/settings.php:166
./admin/upgradesettings.php:58
./lib/adminlib.php:2607:2608

$CFG->AJAXCAPABLE

./course/format/topics/ajax.php:7
./course/format/weeks/ajax.php:7
./course/view.php:153:158

$CFG->AJAXTESTEDBROWSERS

./course/format/topics/ajax.php:8
./course/format/weeks/ajax.php:8
./course/view.php:154:161

$CFG->ALLOWCATEGORYTHEMES

./course/category.php:97
./course/editcategory.php:64
./course/editcategory_form.php:20
./lib/weblib.php:3003

$CFG->ALLOWCOURSETHEMES

./course/edit_form.php:143
./lib/weblib.php:2998
./theme/chameleon/ui/chameleon.php:14
./theme/chameleon/ui/css.php:18

$CFG->ALLOWEMAILADDRESSES

./lib/moodlelib.php:4440:4441:4457

$CFG->ALLOWOBJECTEMBED

./lib/weblib.php:6584:6590

$CFG->ALLOWTHEMECHANGEONURL

./lib/setup.php:599

$CFG->ALLOWUNENROL

./lib/db/access.php:383

$CFG->ALLOWUSERBLOCKHIDING

./blocks/moodleblock.class.php:376
./lib/weblib.php:6479

The post All configuration variables in Moodle code – Part two appeared first on GL.IB.LY.

Searching compressed Apache logs

tariq — Tue, 23 Jun 2009 06:27:23 +0000

From time to time things tend to go wrong and systems tend to go crazy. Sometimes these errors are more of nuisance than anything else, an intermittent annoyance you’d prefer not to investigate — trawling through logs is a pain in the backside. Now you need to take a peak at those compressed Apache error log files, ugh! Well before you look for suitable places to hang some rope try running some expressions across a certain number of files. Normally you’d only want to look at logs for the last 7 days, so lets give that a go.

The error messages I am interested in all contain the text Healthcheck so I will use trusty grepfind matching lines in the logs. I will use find to get me all the files modified in the last seven days. A program called xargs will then be used to call cat which will pipe to a compression program which finally pipes to grep. Put this all together and you get something like this:

find /var/log/apache2 -type f -name "error_*.bz2" -mtime -7 | sort | xargs cat | bunzip2 | grep"Healthcheck"

If your files are tar.gz then you’ll need to replace bunzip2 with gzip -dc. So that would look like:

find /var/log/apache2 -type f -name "error_*.tar.gz" -mtime -7 | sort | xargs cat | gzip -dc |grep "Healthcheck"

Have fun!

The post Searching compressed Apache logs appeared first on GL.IB.LY.

Common translation errors in Moodle language packs

tariq — Fri, 06 Feb 2009 06:34:33 +0000

If you have ever had to deal with language packs you will know how much of a pain they can be. I regularly interact with 9 Moodle language packs and I don’t like doing that much. For some reason, can’t remember why now, we gave people the raw language string files (PHP code files) to translate and they translated these files and sent them back. Only these people are not PHP coders and they make mistakes. Here is my list of what goes wrong and how to solve.

*Incorrect syntax*
They edit the files and send them back to us, but they delete a semicolon or something. Check all language pack files with the following shell script.

#!/bin/bash
for i in $( ls *.php); do
php -l $i
done

On windows try:

@echo off
FOR %%A IN (*.php) DO php -l %%A

*UTF8 Header*
This is prepended to the start of some files by certain editors. As the header comes before the tag the header gets output and this will do all sorts of nasty things like corrupting user sessions. The solution is to strip utf headers from all files.


I use a script to add and strip UTF8 headers from entire directories.
*Translated variable names*

When the translators see something like $a->name they sometimes translate name. Again this isn’t easy to find but I use the following line to get malformed variable names.
grep -nroe “\$a->[^ .\<>/';:\!,\?)(]*” PATH_TO_DIR | grep -v “\$a->[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*”

PATH_TO_DIR/block_class_list.php:14:$a->
PATH_TO_DIR/block_task_list.php:30:$a->
PATH_TO_DIR/block_task_list.php:34:$a->
PATH_TO_DIR/block_task_list.php:34:$a->ì ëª©

This has caught all issues of this kind to date (for me), but it does rely on illegal variable names. A better approach would be to keep a whitelist of variables extracted from a known good language pack. If we find anything that looks like a variable in a another language pack and doesn’t appear on this list we can throw an error.
*Converting files to UTF-8 format*

Sometimes countries (especially those using accented languages) tend to send us ASCII encoded files, these files will have to be converted to UTF-8 format so that accented characters are display correctly.
Here is a one liner to convert all files in the current directory to UTF-8 format:
for f in ./*; do `mv $f $f“.bak”; iconv –from-code iso-8859-1 –to-code UTF-8 $f“.bak” > $f`;done
If you are happy with changes do a
rm *.bak
See also: ISO 8859-1 and Converting files to UTF-8.
*Other things which go wrong*

Translators might change $string['mystring'] = 'Hello $a!'; to $string['mystring'] = "Hello $a!";. Subtly $a will just not appear in the output as the double quoted string will be evaluated before Moodle can swap $a with some dynamic data. The following one liner catches most errors for me even though it isn’t complete.
grep -nro “= \”” PATH_TO_DIR
I guess you could lump all that together in one nice big shell script. If you do send me a copy


The post Common translation errors in Moodle language packs appeared first on GL.IB.LY.



All configuration variables in Moodle code?
tariq — Mon, 15 Dec 2008 07:10:15 +0000
Sometimes Moodle introduces some nice new configuration variables and I like to make sure that I know what they are and where they are. So to begin my investigation without heading off to the web I use the following command to get me a sorted list of the $CFG variables.

grep -r –no-filename -o ‘\$CFG->[a-z][a-z]*’  | sort -u
Good old trusty grep is used once again. We use grep to recursively, -r, look at all the files in the path_to_moodle. We tell grep to not print out the file names that patterns where found in with –no-filename and the -o option tells grep to output the exact pattern matched. The pattern itself is of the form: starts with “$CFG->” and is followed by one, or more lowercase characters. The pipe to sort -u sorts output and removes duplicate strings.
This command will produce something like the following list which I made using Moodle 1.9.2.
$CFG->admin

$CFG->adminblocks

$CFG->admineditalways

$CFG->adminseesall

$CFG->adminusehtmleditor

$CFG->ajaxcapable

$CFG->ajaxtestedbrowsers

$CFG->allowcategorythemes

$CFG->allowcoursethemes

$CFG->allowemailaddresses

$CFG->allowobjectembed

$CFG->allowthemechangeonurl

$CFG->allowunenrol

$CFG->allowuserblockhiding

$CFG->allowusermailcharset

$CFG->allowuserthemes

$CFG->allowvisiblecoursesinhiddencategories

$CFG->allusersaresitestudents

$CFG->alternateloginurl

$CFG->an

$CFG->apacheloguser

$CFG->apachemaxmem

$CFG->aspellextradicts

$CFG->aspellpath

$CFG->assignment

$CFG->attemptuniqueid

$CFG->auth

$CFG->autolang

$CFG->autologinguests

$CFG->backup

$CFG->block

$CFG->blocks

$CFG->blocksdrag

$CFG->blog

$CFG->bloglevel

$CFG->bounceratio

$CFG->cachetext

$CFG->cachetype

$CFG->calendar

$CFG->chat

$CFG->clamfailureonupload

$CFG->config

$CFG->cookiehttponly

$CFG->cookiesecure

$CFG->country

$CFG->coursemanager

$CFG->coursemanagers

$CFG->coursesperpage

$CFG->coursetheme

$CFG->creatornewroleid

$CFG->cronclionly

$CFG->cronremotepassword

$CFG->currenttextiscacheable

$CFG->customersupportemail

$CFG->customersupportemaildisplay

$CFG->customscripts

$CFG->data

$CFG->datadir

$CFG->dataroot

$CFG->dbfamily

$CFG->dbhost

$CFG->dblogerror

$CFG->dbname

$CFG->dbpass

$CFG->dbpersist

$CFG->dbsessions

$CFG->dbtype

$CFG->dbuser

$CFG->debug

$CFG->debugdisplay

$CFG->debugsmtp

$CFG->decsbureauid

$CFG->decsitemtypeid

$CFG->defaultallowedmodules

$CFG->defaultblocks

$CFG->defaultcourseroleid

$CFG->defaultfrontpageroleid

$CFG->defaultrequestcategory

$CFG->defaultuserroleid

$CFG->deleteunconfirmed

$CFG->denyemailaddresses

$CFG->detect

$CFG->digestmailtime

$CFG->digestmailtimelast

$CFG->directorypermissions

$CFG->dirroot

$CFG->disablebyteserving

$CFG->disablecourseajax

$CFG->disableglobalshack

$CFG->disablegradehistory

$CFG->disablemycourses

$CFG->disablescheduledbackups

$CFG->disablestatsprocessing

$CFG->disableupgradelogging

$CFG->disableuserimages

$CFG->displaydebug

$CFG->displayloginfailures

$CFG->docroot

$CFG->doctonewwindow

$CFG->eaccelerator

$CFG->editorbackgroundcolor

$CFG->editordictionary

$CFG->editorfontfamily

$CFG->editorfontlist

$CFG->editorfontsize

$CFG->editorformatlist

$CFG->editorhidebuttons

$CFG->editorkillword

$CFG->editorspelling

$CFG->editorsrc

$CFG->emailconnectionerrorsto

$CFG->emoticons

$CFG->enableajax

$CFG->enablecourserequests

$CFG->enableglobalsearch

$CFG->enableglobalshack

$CFG->enablegroupings

$CFG->enablehtmlpurifier

$CFG->enableoutcomes

$CFG->enablerecordcache

$CFG->enablerssfeeds

$CFG->enablestats

$CFG->enabletrusttext

$CFG->enrol

$CFG->errordocroot

$CFG->exercise

$CFG->extendedusernamechars

$CFG->extratabs

$CFG->filelifetime

$CFG->filter

$CFG->filterall

$CFG->filtermatchoneperpage

$CFG->filtermatchonepertext

$CFG->filteruploadedfiles

$CFG->footer

$CFG->footerlinks

$CFG->forcefirstname

$CFG->forcelastname

$CFG->forcelogin

$CFG->forceloginforprofiles

$CFG->forcetimezone

$CFG->forgottenpasswordurl

$CFG->formatstring

$CFG->formatstringstriptags

$CFG->forum

$CFG->framename

$CFG->frametarget

$CFG->frontpage

$CFG->frontpageloggedin

$CFG->fullnamedisplay

$CFG->gdversion

$CFG->geoipfile

$CFG->glossary

$CFG->googlemapkey

$CFG->grade

$CFG->gradebookroles

$CFG->gradeexport

$CFG->gradehistorylifetime

$CFG->gradepublishing

$CFG->guestloginbutton

$CFG->guestroleid

$CFG->handlebounces

$CFG->header

$CFG->hiddenuserfields

$CFG->hideactivitytypenavlink

$CFG->hivecbid

$CFG->hivehost

$CFG->hivepassword

$CFG->hivepath

$CFG->hiveport

$CFG->hiveprotocol

$CFG->hiveusername

$CFG->hotpot

$CFG->hotpotismobile

$CFG->hotpotroot

$CFG->hotpottemplate

$CFG->htmleditor

$CFG->httpsthemewww

$CFG->httpswwwroot

$CFG->ignoresesskey

$CFG->intcachemax

$CFG->iplookup

$CFG->javascript

$CFG->journal

$CFG->keeptagnamecase

$CFG->lams

$CFG->lang

$CFG->langcache

$CFG->langlist

$CFG->langmenu

$CFG->lastexpirynotify

$CFG->lastnotifyfailure

$CFG->latinexcelexport

$CFG->ldap

$CFG->libdir

$CFG->local

$CFG->locale

$CFG->loginaspassword

$CFG->loginhttps

$CFG->loglifetime

$CFG->logsql

$CFG->longtimenosee

$CFG->maildomain

$CFG->mailnewline

$CFG->mailprefix

$CFG->max

$CFG->maxbytes

$CFG->maxeditingtime

$CFG->memcachedhosts

$CFG->memcachedpconn

$CFG->message

$CFG->messagewasjustemailed

$CFG->messaging

$CFG->migrated

$CFG->minbounces

$CFG->minpassworddigits

$CFG->minpasswordlength

$CFG->minpasswordlower

$CFG->minpasswordnonalphanum

$CFG->minpasswordupper

$CFG->mnet

$CFG->moddata

$CFG->modpixpath

$CFG->my

$CFG->mymoodleredirect

$CFG->noconvertjournals

$CFG->nodefaultuserrolelists

$CFG->noemailever

$CFG->nofixday

$CFG->nolastloggedin

$CFG->nonmetacoursesyncroleids

$CFG->noreplyaddress

$CFG->notifyloginfailures

$CFG->notifyloginthreshold

$CFG->notloggedinroleid

$CFG->old

$CFG->opentogoogle

$CFG->os

$CFG->ostype

$CFG->pagepath

$CFG->pagetheme

$CFG->passwordpolicy

$CFG->passwordsaltmain

$CFG->pathtoclam

$CFG->pathtodu

$CFG->perfdebug

$CFG->pixpath

$CFG->preferlinegraphs

$CFG->prefix

$CFG->preventaccesstohiddenfiles

$CFG->protectusernames

$CFG->proxyhost

$CFG->proxypassword

$CFG->proxyport

$CFG->proxytype

$CFG->proxyuser

$CFG->qtype

$CFG->quarantinedir

$CFG->quiz

$CFG->rcache

$CFG->rcachettl

$CFG->recaptchaprivatekey

$CFG->recaptchapublickey

$CFG->registerauth

$CFG->registered

$CFG->release

$CFG->repository

$CFG->repositoryactivate

$CFG->repositorywebroot

$CFG->resource

$CFG->resourcetrimlength

$CFG->respectsessionsettings

$CFG->restrictbydefault

$CFG->restrictmodulesfor

$CFG->rolesactive

$CFG->runclamonupload

$CFG->running

$CFG->scorm

$CFG->search

$CFG->session

$CFG->sessioncookie

$CFG->sessioncookiepath

$CFG->sessiontimeout

$CFG->showblocksonmodpages

$CFG->showcrondebugging

$CFG->showcronsql

$CFG->siteblocksadded

$CFG->siteidentifier

$CFG->sitemailcharset

$CFG->sitepolicy

$CFG->slasharguments

$CFG->smartpix

$CFG->smtphosts

$CFG->smtpmaxbulk

$CFG->smtppass

$CFG->smtpuser

$CFG->sso

$CFG->statscatdepth

$CFG->statsfirstrun

$CFG->statsmaxruntime

$CFG->statsrolesupgraded

$CFG->statsruntimestarthour

$CFG->statsruntimestartminute

$CFG->statsuserthreshold

$CFG->stringfilters

$CFG->stylesheets

$CFG->supportemail

$CFG->supportname

$CFG->supportpage

$CFG->tagsort

$CFG->template

$CFG->textfilters

$CFG->textfilterx

$CFG->theme

$CFG->themedir

$CFG->themelist

$CFG->themeorder

$CFG->themewww

$CFG->timezone

$CFG->tracksessionip

$CFG->type

$CFG->unicodecleanfilename

$CFG->unicodedb

$CFG->unzip

$CFG->upgrade

$CFG->usepaypalsandbox

$CFG->usesid

$CFG->usetags

$CFG->version

$CFG->wiki

$CFG->wordlist

$CFG->workshop

$CFG->wwwdir

$CFG->wwwroot

$CFG->xml

$CFG->xmldbdisablecommentchecking

$CFG->xmldbdisablenextprevchecking

$CFG->xmldbreconstructprevnext

$CFG->xmlstrictheaders

$CFG->zip
The post All configuration variables in Moodle code? appeared first on GL.IB.LY.