Categories
Programming

Learning Python

Here are some notes from class the last few weeks (reverse order sorry!):

MONDAY 25/11/19

Today the topic was dealing with filenames and understanding some of the finer details of real world coding. I enjoyed using the Python console – it felt very raw compared to Pycharm! We used the OS module today which I discovered in relation to a previous task (mentioned in journal). Makes you realise how easily code can be broken and how many considerations or backup plans need to be written. At work I have learnt a lot of what it means to make my concepts production ready or at least ready for testing on our system.

I enjoyed the walk-through nature of the lecture. The most helpful aspect is hearing an expert think out loud. The confidence with which they take certain steps or the importance they place on certain points are very telling. To me it means, ok this you will start doing quickly with no problem whereas this other skill will always have the opportunity to go wrong for example.

I also didn’t realise there was quite so many modules in Python, numbering into the thousands! So of course it’s impossible to memorise such a grand subject. Stackoverflow was highlighted and in my job as well I’ve seen how useful that is. Personally I find the discussion and debate that often occurs around various solutions very valuable as it educates you to see how and why it could be done a certain way. But I’ve also been cautioned to only implement what you understand – to copy and paste is very bad practice and can lead to you adopting bad habits and design. Also I use it as a last resort – I think engaging your own logic to its full extent is vital.
Despite being able to follow the steps in the lecture, the actual syntax confused me a little bit when I sat down to write with it. It really is a big jump to go from understanding code, to being able to write it yourself with different parameters. I mean that even for a small module like this. I underappreciated that very much before starting this class.

N.B I came across this function while researching maketrans: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.str.translate.html. Looks like this has very smilar funcationality in a library (Pandas) I’ve seen mentioned countless times in data science articles, so delighted to be using something of key value in the future!

TUTORIAL
1) Made another newbie error and ended up with endless printout of an 8ball answer! I corrected my logic and completed the task without taking too much time.
2) a)Really helpful to have the walk through guide but more so to actually do it and not just assume it would be no problem – learning is doing after all! Two mistakes here. First I used the Python IDLE shell instead of the interpreter accessed via Pycharm and secondly I didn’t realise the chdir function was not ‘inline’ and thus not being called! The hidden message : “Nobody expects the Spanish Inquisition. ” And, “It is a mistake to think that you can solve any major problem just with potatoes.”
b) The solution here is the reverse, to add a random number. This page (below) helped me choose the right function from many random generator functions that Python has to offer: randint(). (https://docs.python.org/3/library/random.html?highlight=randint#random.randint > it looks to have a popular use case for random sampling). The function itself provided a nice example of using for and in for iterating over a sequence.
c) I was getting this error:
line 19, in encode_files
number = str(random.randint(99, 999)) #random = anystring, parameters represent lower and upper limits
NameError: name ‘random’ is not defined
Before realising I needed to import the random module! This then worked as hoped as shown in the screenshot below. To enable faster progress I skipped the manual process of creating a dictionary to create my own actual secret message now that I have a working methodology. I hope to come back to the advanced tutorial.

******************************************************************************************************
TUESDAY 26/11/19

This lecture takes our manipulation of files to a step more complex, reading and writing to the file to alter content. I read a little on the Unicode website which at first looks like an emoji repository! I liked this information and how it relates to software becoming truly global :

“The emergence of the Unicode Standard and access to tools supporting it are among the most significant recent global software trends.> Each character in a language is assigned a unique code. Check out the complete list (warning: there are close to 150,000 and counting!).”

This extra information (https://docs.python.org/2.3/whatsnew/section-slices.html) helped provide some further examples of advanced slicing. I think I will be implementing it on our Sudoku assignment so I will spend extra time on this “step” or “stride” argument as they call it officially.

We must specify the file mode at the end of an open request for files. Some of the functionality here seem very similar to my initial trial of the NLTK (https://www.nltk.org/book/) library for NLP last year. Will research here (https://docs.python.org/3/library/codecs.html) to understand what we can use to secure files. It echoes some of the work we did in our Intro to Security module a few weeks ago. I didn’t know that there was cryptography built into the core of Python! (“The secrets module is used for generating cryptographically strong random numbers suitable for managing data such as passwords, account authentication, security tokens, and related secrets.“ I read on Stackoverflow that many developers still naively use the random number module instead of secrets, but that’s of course one individuals observation.

I see that some of the topics covered today are highly relevant to our upcoming assignment, so I’ve done some extra practice and reading up of examples just to try and help solidify my understanding. I have two main observations about coding so far:
1) It is quite easy to summarise an area of the language and provide a use case
2) It is anything but easy to grasp the complexity and depth there is to applying a feature in a few different use cases!

TUTORIAL

1) Using a multi line quotes, I realised I had to use triple quotes for the string to print out as desired.

2) At first I used the reversed() function but this didn’t work as expected. It printed out a long object reference. Referring to the slides showed I should use the method [::-1], with the semi colons implying the full length of the string and the -1 defining the direction of the iteration. Looking at the suggested solution afterwards also helped me learned usage of the strip() method, removing whitespace. I also noted the removal of capitalisation using upper(), I had forgotten to include this.

At first I thought this was simple and quickly applied the technique of the previous question but I ended up with a mirror effect rather than a reversal as below! After some consideration I realised this is due to the interpretation of the string as individual characters instead of the whole world. It’s obvious yet incredible how precise you have to be.

To get to my solution I read over the slides and found (extra information from the docs (https://docs.python.org/3/library/stdtypes.html?highlight=split#str.split)) the split method which keeps the whole word intact. Further research with Googles aid led me to this article (https://medium.com/@hmurari/a-popular-programming-interview-question-reverse-words-of-a-sentence-3bac606d15a2) which provided a wonderful level of insight into how many angles/opportunities/variations there are to everything we are learning! It gave me a way of using the reversed function I initially tried to use while giving a cleaner print result. Also interesting to see that this small question is common in job interviews (according to the author!). I noticed the suggested Noroff solution used the end keyword. I found this article helpful on understanding the implementation (https://www.journaldev.com/15182/python-print).

4) I continued on in my file w4_d2_1. I added some text to the file as below. Our example in the lecture demonstrated a slightly complex iterative ability whilst I achieved the below by simply including the text string in the statement. Use of append is not even needed to do this.

5) This activity and section of the lecture I liked, especially after our introduction to security in computing earlier in the semester. To enable encoding to another file I simply reset the variable file_name.

I’m sure there is a much more ‘Pythonic way’ to write this. (In fact I’m well aware of the novice status I currently have so I assume everything is a long way from professional. ) But speaking with my boss at work I’ve taken heart from knowing that getting the desired result in whichever way is valuable, but secondly, even if it is long from professional there is value in producing something readable where the logic can be followed. So perhaps 13 lines isn’t efficient or stylish but it works! I’m very much looking forward to feedback in this regard as well as simply being exposed to more and better code.
c) So I just had another incredibly long read of one incredibly long thread on Stack Overflow. There are so many tools available to take care of so many needs. This (https://stackoverflow.com/questions/3678869/pythonic-way-to-combine-two-lists-in-an-alternating-fashion) was very informative yet not something I’m ready to apply yet. “itertools – Functions creating iterators for efficient looping” seems to be a core part of Python (https://docs.python.org/3/library/itertools.html) I wonder how long it will take me to get to this stage? In my actual solution I tried to employ something readable, simple and of course something we have done already with the list selection syntax [::n]. It took a bit longer to implement because I didn’t realise that although the file was a list type it was coming as a list of one element of a long string so it needed to have the split function applied first, to create multiple elements.

7) I was able to re-use a lot of elements to achieve the desired results. I feel like there are probably a few edge cases that need to be considered for a really robust solution, something out of todays scope. For example, I originally downloaded the Iliad by Homer and it failed to work because of the symbols/character types. I guess a unicode based solution would work well compared to my reading from a txt converted file,

Here are the requested results:

Plato:
‘The’ = 2060 times
‘The ’ – with a space = 1052 times
then = 58 times
there = 87 times

*After including the lower and strip methods, my ‘the’ count increased to 2238!

James Joyce:
‘The’ = 1388 times
‘The ’ – with a space = 1052 times
then = 34 times
there = 46 times

*With lower and strip methods applied

********************************************************************************************************************************
WEDNESDAY 27/11/19

LECTURE NOTES
Today’s lecture focuses on references and modules. Our key goal is to avoid spaghetti code! We will learn how to package code in a readable and usable format and how information works in the background. When I first skimmed over the slides before the class I was pleasantly surprised to see another topic which looks to put us in great shape for working out in the real world. The surprise was also due to the very detailed or ‘deep’ nature of the topic – something I hadn’t seen mentioned in online courses before.

For me this is the key phrase – “Variables hold references to data stored in memory –not the data themselves. Functions retain variable reference on function stack.”
Reading that sentence on it’s own isn’t as helpful as hearing in the lecture about the further complications of it. It appears that nothing is black and white in programming! There is always an edge case that breaks your script! I have heard of unit testing and have read some demonstrative code which I guess can mitigate issues similar to that mentioned in the lecture, so a check that the object id matches could be of value. We are told that it is when working with collections that we need to pay the greatest attention for this kind of error. It’s of further importance to note that within a function there is a difference in the way that immutable and mutable types behave, ‘if we pass mutable arguments…. they can be changed in place in the function.’

I’ve read further in the area here (https://docs.python.org/3/library/stdtypes.html) and this also led me to discover a little further in the docs > https://docs.python.org/3.8/library/collections.html. I had seen the ‘deque’ function mentioned in forums before as a very efficient way of doing things but I guess using it will have to wait for now as a beginner.

I’d often read that Python is the best first language to learn and when discussing the call-by-object reference I learned one of the reasons why other than the clearly simpler syntax. I’ve seen the code required just to write hello world in Java so I’m happy with Python for now!

public class HelloWorld {

public static void main(String[] args) {
// Prints "Hello, World" to the terminal window.
System.out.println("Hello, World");
}

}

Having completed the previous weeks assignments I feel comfortable with the fundamentals of creating ones own module and importing it in a different file. The key for simplicity now is that we do this from the same folder location. But it was still super valuable to go over the case of priority and overwriting values. This is sure something that will catch us out in the future. The key advice was “Rename functions on import, to avoid using dot-suffix notation and reduce chance of inadvertent function overriding”.

Perhaps in a future role of employment it will pay huge ‘efficiency’ dividends to create your own modules, highly specific to your tasks. I’ve read how much skilled software engineers detest repetition or a lack of automation. Hopefully I can create similar tools or effects to those I’ve seen discussed in the future with an improved skill set!

**************************

DAY 7 : LECTURE NOTES & REFLECTION
We received some information our assignment on the Sudoku game. Basically today’s lecture will cover some of the fundamental methods needed for building it and that it is a test of our logic and construction so there is no need to build a GUI.

LISTS
We began with an introduction to the list type and I liked the clear visuals used in the slides. The Udemy course covered this type and I feel it really helped me understand the way that one counts in code with the first position being zero. In the intervening time further reading has also helped me understand the real world application of the various use cases rather than simply knowing the process in the abstract. In my new job I’ve begun working with some data in the list format but the section is in plain JS so I hope I don’t mix the syntax up too often!

In Javascript (and many other languages Rayne noted) the list type is referred to as an array. A list can be thought of as a large container, full of smaller containers. Another analogy Rayne provided was that it is an organised system, like a library. She asked to investigate negative integers and I found this thread to be helpful while investigating (https://stackoverflow.com/questions/2451776/working-with-negative-numbers-in-python). But I think the main use case is counting backwards through a list. If an index has a negative value, it counts backward from the end of the list. -1 is the last element of the list, -2 is the second to last etc.

OPERATIONS

  • Use the [] notation
  • Position 1 is 0 – counting in code is different
  • List types positively affects memory storage

Other common operations include checking the object type and using the ‘in’ function which outputs true or false logic.

#check type
print(type(nums)) = <class ‘list’> #checks what nums type is
print(type(nums[2])) = <class ‘int’> #checks what type the the third object in the list is

Also you can search for something and check how many times it occurs.

print(1 in nums) #true or false logic, if true print for example
print(nums.count(3)) #how many occurrences are there of the int 3

Whilst iterating through a list you can use ‘for’ and ‘in’. For enables an action to be taken. It is important to declare a variable first with some value, maybe 0. At this point we were recommended to re-code / write for ourselves, everything we are introduced to. It serves as a mini test as to whether it has been internalised rather than simply taken in passively. I love the applied nature of the class and it necessitates a similar type of learning and reflection to this journal. For me the extended repetition is the only way to learn the syntax.

OTHER LIST MANIPULATIONS
You can use the len (for checking length) and del (delete) functions over the whole list or an item in the list. You can also select for sections of the list using the syntax:
del nums[0:3]
This means from position 0 up to position 3 – it won’t delete position 3.
You can also do this in reverse with a negative number. Another feature of lists is that they can be nested ie you can have a list of lists!
Append is self explanatory and apparently one of the most common functions used. Pop is its opposite, removing things from a list. This is almost identical to the delete function apart from the opportunity it offers to work with it one more time by returning it after the operation.

Not discussed was the remove function which does so by deleting a matching element wheres the others delete by index position. You can also empty a list using the clear() function, assigning [] to the list or by using *=0. The function reverse() does the obvious, swapping the order. It seems the multiple ways of doing similar things are to accommodate older versions of Python as there is so much in production.

The notation list1[3][4] accesses index 3 and the fourth element of that object. Ideal for working with nested lists. We were given the orchard example which works like a graph with x and y axis. It’s possible to add a third value here which could represent the orchards in different years. Otherwise known as a multidimensional list.

COMMON MISTAKES
The following was highlighted:
Important to note that parameters can’t be referenced, you need to use index positions to work with objects.
Must use append rather than mathematical operator in your code as working with lists entails a specific type of concatenation.
Lists are mutable (can be changed and worked with).
Recommended syntax is over multiple lines for clear readability.
Remember to comment in a succinct yet fully explainable way – there is a huge chance you will forget where your thinking was directed in the future – especially if you develop your skills.

ERRORS
When trying the code in the slides it resulted in
1)line 3, in <module>
del(basket[‘milk’])
TypeError: list indices must be integers or slices, not str
>>>need to use index position
2)print(‘amount of milk’, basket.count[‘milk’])
TypeError: ‘builtin_function_or_method’ object is not subscriptable
need to use round brackets
3)del basket[item]
TypeError: list indices must be integers or slices, not str
>>>need to use index position
4) basket = basket + ‘cider’
TypeError: can only concatenate list (not “str”) to list
Need to use append function instead

READING
As prompted I read through the documentation on strings (https://docs.python.org/3/library/string.html). In addition I found this documentation very helpful (https://docs.python.org/3/howto/sorting.html). I could obviously write extensively on the topics and all there various implementations but as time is of a premium my focus is very much on the fundamentals. I think being curious also makes it very easy to get distracted by finding out about areas of Python that are not the most relevant. In a simple way, simply being exposed to the terminology and the methods make everything more familiar.

TUTORIAL
1) Although the majority of this lecture is material I’ve covered before, I’m not naive enough to think its automatic, intuitive or internalised for me. It was a good refresher to play around with some of the functions again and be further acquainted with PyCharm having used Jupyter notebooks before.

2) A familiar pattern here! Took way longer than first expected. I feel like I have a clear understanding of the logic but my novice ability prevents speedy execution. I correctly selected a while loop and the append function from the outset by I made the mistake of not using an over writable parameter,

3) Hopefully these are not mistakes I will repeat – it’s so obvious in hindsight! This time I wondered why I could only get a True output. It was due to my input type not being list thus everything was simply one value – there was only one index, so it couldn’t be un-ordered! Happy to get it working now. It seems good practice to print out inputs as a way of double checking what the program is doing.

4) I thought I had the solution quickly but I can’t get my interpretation to work on multiple occurrences of duplication, it always leaves two duplicates even though the code loops over deleting any object with count value higher than 1. I discovered the conversion to a set is by far the simplest solution although you lose the list order. Another solution is filling an empty list with the first instance of each item in the original list.

5) Another rude awakening after what seemed very logical when a similar example was discussed in the lecture! I like to try and solve the questions totally on my own at first without any reference to the slides or the Python docs. The mistake this led to was to spread the row data over separate lists instead of one extensive list(helps to read the question!!). This isn’t totally wrong but is more suitable for multiple orchards or years/seasons rather than rows.

I had seen the suggestion of using sep=“\n” to output the data on different lines but I was able to discover a simpler method using the elem function. The next task was straight forward but I struggled next with changing a value, getting the “IndexError: list index out of range“ result, using both remove and pop functions. I assumed the solution was more complicated than simply re-assigning the value! Following that a similar methodology outputted the total using the sum function. I didn’t have time to code up the extra challenges but it looks like a simple math operation of i*1.2 would change the total yield and the append function would create extra rows.

********************************************************************

DAY 8 : RECAP
This is the first lecture I’ve watched from Kristiansand. Will be interesting to see how the transfer of ideas/info will be with a different style. First impressions are positive!

The lecture kicked off with a recap of what we should feel some confidence with now. This included that lists are mutable, we can edit them extensively – they are very dynamic and thus a very popular part of Python. There was then a quick discussion on personal preferences for using for or while loops. I must say I’ve noticed the crossover in utility of these methods and often in forum discussions (ex StackOverflow) it comes down to what is more pythonic! Which I guess means a better solution from the experts angle. We were prompted to look at the Google style guide for writing in Python for our assignment so there is perhaps something to focus on there. I personally liked the point of thinking of this in terms of start and stop or count rather than the abstract name of the control structure.

ENUMERATE()
I researched the function here (https://docs.python.org/3/library/functions.html#enumerate) and I must add it’s nice to understand a little after seeing it in my searching. It “returns a tuple containing a count“ which enables a programmer to keep track of the number of iterations during loops. The output of the function is a tuple which means it cannot be modified.

CONTD LISTS
Next up there was a summary of accessing parts of lists. I feel like I learned so much from doing things wrongly in this area with the previous activity! I’m also conscious that these are very small pieces of data we are working with. It’s intimidating to think of potentially working with large and complex data sets! Was interesting to learn about the ability to assign multible variables in one line of code. From what I read, conciseness and efficiency are the holy grail of quality code so I guess this will help in that regard.

TUPLES
Some programs deal with or generate data that should not have an ability to be changed such as exam results. Further reading shows that this can also reduce the potential for code error in larger scripts and manage memory more effectively. This data types are called tuples and are thus defined as immutable. They are declared with round rather than square brackets. I found this section in the official docs enlightening:
“on output tuples are always enclosed in parentheses, so that nested tuples are interpreted correctly; they may be input with or without surrounding parentheses, although often parentheses are necessary anyway (if the tuple is part of a larger expression). It is not possible to assign to the individual items of a tuple, however it is possible to create tuples which contain mutable objects, such as lists.”

SETS
Sometimes the functionality required is knowledge of unique values – this is known as a set. Methods we can use include similar abilities to that we have tried in working with list values but the new types include intersection, union and difference. If you try and add a duplicate value it is simply ignored. {} curly braces can be used in set notation but not in creation as this defines a dictionary type. The update function treats each character individually and so will display a word such as ‘cat’ as ‘c’,‘a’,‘t’. It’s not intuitive to me at all! Sets are unordered and so cannot be accessed by index.

DICTIONARIES
Using the {} notation these are known as key value pairs. The syntax is dict = {‘user’: ‘oscar’, ‘weather’:‘cold’}. The values on the left are known as keys and can be accessed directly with dictdemo.keys(). The same is true for the values on the right hand side dictdemo.values(). I feel like I’m wise to the fact now that although this seems very logical and I’m happy with what we are being told, remembering the options I have when in the IDE will not come quickly. I’m going to need that error callback help!
A major function is searching for items using dictionaries as one can easily access/iterate through a set of users for example. A key value (no pun intended!) is the speed and efficiency of this search model. The example of search also reminded us how flexible Python is.

Logically converting the string input to all lowercase using the .islower() notation would be appropriate but I came across the casefold() function which works even better when considering foreign language inputs. Because it actions in the function the original string is not altered. This is use of a local variable, a temporal variable as far as I’m aware. I’ve allowed for potential mismatch on both the word and letter input.

SECTION 2
1) I remembered that the pop function also returned the selected item, so I stored that as a variable at first pass. But I realised that would alter the original list and potentially involve more memory/time. So I did the more obvious solution of simply reading the index. To get the last item I used the negative counting syntax as it was simpler than finding the length of the list.

2) I used the operation that according to our lecture isn’t so popular unless working with math operations, intersection and difference. Working with sets instead of lists is a little different and I struggled with this initially when looking for objects in the user input section. It’s so easy to go through these tutorials and think it will be no problem but there is such precision required from the interpreter!

3) Having practiced dictionaries before and having a simple lookup logic made this a quick task for me. I look forward to working more with this type format and extending the functionality even more to get closer to replicating something in the real world.

========================================

DAY 9
Wednesday was a Q&A session, offered due to popular demand based on previous years feedback. New to actually writing code, its always helpful to hear further explanation of topics but also to see what commonly is not intuitive.

As a net student I wasn’t able to join live for the class. I feel like if I was there in person I might interact substantially more. On the other hand, the live streaming , the permanency of the video probably hinders the interaction. I guess its one of the challenges of net study – something has to give when there is such flexibility with net study.

I note on the Skype chat there was a lot of frustration , a lot exclamation and question marks! My thoughts are that first, understanding the context of what we are learning can be very helpful. For example how the dictionary type helps search and how that perhaps maps to a frontend app. Secondly, relating to being a professional competitor and teacher, I know that learning something new from the ground up is ugly and hard. You might say you just have to keep looping over areas until you see the pattern! Thirdly I think the higher a bar to entry that something has, the more value it provides. The investment of time and energy ensures a commitment that leads to a feeling of mastery. I’ve experienced this recently in learning Norwegian but lets just say I still have a way to go!

In summary of this week I have to say it was really tough on time, a familiar comment from me! Part of me wonders about what the luxury of full time study in my younger years (without a family!) would have felt like! But I kinda like the extra challenge.

My new job has me working at Greps for Gunnar Bergersen, an associate professor of ‘informatics’ and computing at Oslo University. It’s an amazing opportunity to engage with him on my progress. It’s been very exciting to have the chance to already apply some of what we have learned/are learning in the real world. I’ve already been exposed to some data sets in the role and I look forward to being able to interact with them and their bigger counterparts even more so.

I can conclude this post with a comment on my main goal with programming. I’m aware of how vast and dynamic the landscape is, so rather than aiming for mastery of a language I have the goal of thinking, planning and doing tasks more logically. To more quickly see the bigger picture, to see secondary effects. It is so very motivating to move into a new field and try and find a way to become effective.

LECTURE NOTES & REFLECTION
We received some information our assignment on the Sudoku game. Basically today’s lecture will cover some of the fundamental methods needed for building it and that it is a test of our logic and construction so there is no need to build a GUI.

LISTS
We began with an introduction to the list type and I liked the clear visuals used in the slides. The Udemy course covered this type and I feel it really helped me understand the way that one counts in code with the first position being zero. In the intervening time further reading has also helped me understand the real world application of the various use cases rather than simply knowing the process in the abstract. In my new job I’ve begun working with some data in the list format but the section is in plain JS so I hope I don’t mix the syntax up too often!

In Javascript (and many other languages Rayne noted) the list type is referred to as an array. A list can be thought of as a large container, full of smaller containers. Another analogy Rayne provided was that it is an organised system, like a library. She asked to investigate negative integers and I found this thread to be helpful while investigating (https://stackoverflow.com/questions/2451776/working-with-negative-numbers-in-python). But I think the main use case is counting backwards through a list. If an index has a negative value, it counts backward from the end of the list. -1 is the last element of the list, -2 is the second to last etc.

OPERATIONS

  • Use the [] notation
  • Position 1 is 0 – counting in code is different
  • List types positively affects memory storage

Other common operations include checking the object type and using the ‘in’ function which outputs true or false logic.

#check type
print(type(nums)) = <class ‘list’> #checks what nums type is
print(type(nums[2])) = <class ‘int’> #checks what type the the third object in the list is

Also you can search for something and check how many times it occurs.

print(1 in nums) #true or false logic, if true print for example
print(nums.count(3)) #how many occurrences are there of the int 3

Whilst iterating through a list you can use ‘for’ and ‘in’. For enables an action to be taken. It is important to declare a variable first with some value, maybe 0. At this point we were recommended to re-code / write for ourselves, everything we are introduced to. It serves as a mini test as to whether it has been internalised rather than simply taken in passively. I love the applied nature of the class and it necessitates a similar type of learning and reflection to this journal. For me the extended repetition is the only way to learn the syntax.

OTHER LIST MANIPULATIONS
You can use the len (for checking length) and del (delete) functions over the whole list or an item in the list. You can also select for sections of the list using the syntax:
del nums[0:3]
This means from position 0 up to position 3 – it won’t delete position 3.
You can also do this in reverse with a negative number. Another feature of lists is that they can be nested ie you can have a list of lists!
Append is self explanatory and apparently one of the most common functions used. Pop is its opposite, removing things from a list. This is almost identical to the delete function apart from the opportunity it offers to work with it one more time by returning it after the operation.

Not discussed was the remove function which does so by deleting a matching element wheres the others delete by index position. You can also empty a list using the clear() function, assigning [] to the list or by using *=0. The function reverse() does the obvious, swapping the order. It seems the multiple ways of doing similar things are to accommodate older versions of Python as there is so much in production.

The notation list1[3][4] accesses index 3 and the fourth element of that object. Ideal for working with nested lists. We were given the orchard example which works like a graph with x and y axis. It’s possible to add a third value here which could represent the orchards in different years. Otherwise known as a multidimensional list.

COMMON MISTAKES
The following was highlighted:
Important to note that parameters can’t be referenced, you need to use index positions to work with objects.
Must use append rather than mathematical operator in your code as working with lists entails a specific type of concatenation.
Lists are mutable (can be changed and worked with).
Recommended syntax is over multiple lines for clear readability.
Remember to comment in a succinct yet fully explainable way – there is a huge chance you will forget where your thinking was directed in the future – especially if you develop your skills.

ERRORS
When trying the code in the slides it resulted in
1)line 3, in <module>
del(basket[‘milk’])
TypeError: list indices must be integers or slices, not str
>>>need to use index position
2)print(‘amount of milk’, basket.count[‘milk’])
TypeError: ‘builtin_function_or_method’ object is not subscriptable
need to use round brackets
3)del basket[item]
TypeError: list indices must be integers or slices, not str
>>>need to use index position
4) basket = basket + ‘cider’
TypeError: can only concatenate list (not “str”) to list
Need to use append function instead

READING
As prompted I read through the documentation on strings (https://docs.python.org/3/library/string.html). In addition I found this documentation very helpful (https://docs.python.org/3/howto/sorting.html). I could obviously write extensively on the topics and all there various implementations but as time is of a premium my focus is very much on the fundamentals. I think being curious also makes it very easy to get distracted by finding out about areas of Python that are not the most relevant. In a simple way, simply being exposed to the terminology and the methods make everything more familiar.

TUTORIAL
1) Although the majority of this lecture is material I’ve covered before, I’m not naive enough to think its automatic, intuitive or internalised for me. It was a good refresher to play around with some of the functions again and be further acquainted with PyCharm having used Jupyter notebooks before.

2) A familiar pattern here! Took way longer than first expected. I feel like I have a clear understanding of the logic but my novice ability prevents speedy execution. I correctly selected a while loop and the append function from the outset by I made the mistake of not using an over writable parameter,

3) Hopefully these are not mistakes I will repeat – it’s so obvious in hindsight! This time I wondered why I could only get a True output. It was due to my input type not being list thus everything was simply one value – there was only one index, so it couldn’t be un-ordered! Happy to get it working now. It seems good practice to print out inputs as a way of double checking what the program is doing.

4) I thought I had the solution quickly but I can’t get my interpretation to work on multiple occurrences of duplication, it always leaves two duplicates even though the code loops over deleting any object with count value higher than 1. I discovered the conversion to a set is by far the simplest solution although you lose the list order. Another solution is filling an empty list with the first instance of each item in the original list.

5) Another rude awakening after what seemed very logical when a similar example was discussed in the lecture! I like to try and solve the questions totally on my own at first without any reference to the slides or the Python docs. The mistake this led to was to spread the row data over separate lists instead of one extensive list(helps to read the question!!). This isn’t totally wrong but is more suitable for multiple orchards or years/seasons rather than rows.

 
I had seen the suggestion of using sep=“\n” to output the data on different lines but I was able to discover a simpler method using the elem function. The next task was straight forward but I struggled next with changing a value, getting the “IndexError: list index out of range“ result, using both remove and pop functions. I assumed the solution was more complicated than simply re-assigning the value! Following that a similar methodology outputted the total using the sum function. I didn’t have time to code up the extra challenges but it looks like a simple math operation of i*1.2 would change the total yield and the append function would create extra rows.



TUESDAYS LECTURE/ACTIVITY

Yesterday’s lecture and tutorials concerning UML is still on my mind. I feel like I created sub optimal solutions by first using the wrong tool and secondly going straight to a near final copy instead of simply writing a list and iterating from there. The next time I create a UML diagram I intend to use Google Draw which will enable much neater and faster drawing of all the directional lines. Also after handing in the task a few further dependencies and ideas occurred to me but it’s perhaps to granular a level for the task and time plays a factor also. What’s personally motivating is that I will be able to use this method at my new work instead of just sketching lists in a general fashion.

LECTURE NOTES

Despite looking through the notes before the lecture and later in the day (while commuting), it’s still taken me a couple extra days to complete the tasks and collate my thoughts (no excuses – just a hectic week). Watching the lecture, a coherant one, obviously adds a lot of value to the attempted osmosis of new information and concepts. A verbal, concise explanation of the code has helped me.

So in this lecture we learnt some new syntax and how we use def to create our own functions. This allows to abstract processes and create clear logic. It makes it simpler and faster to write good code as we can re-use other code and simply call a function rather than re-write it out again. Rayne highlighted the importance of a few factors when defining ones own function:

  • very important to use the right data type
  • need to be explicit when dealing with data parameters and order in lists
  • understand the difference in global versus local variables
  • understand memory function – stack is temporary and heap is more permanent
  • default values can be provided for certain parameters (I imagine this to be extremely common in building form functions ie pre-selecting country)

With regards to memory management we also discussed that its possible to import single functions and not simply the whole module. Our use case was selecting the pi function from the math module with the syntax math.pi reduced to simply pi. This article helped me get a fuller understanding (https://www.digitalocean.com/community/tutorials/how-to-import-modules-in-python-3).

TUTORIAL REFLECTION

Writing the area finder functionality was interesting as it’s such a simple concept to explain verbally but so much more demanding to explain it in ‘computer language’! You have to be so explicit, so ordered and logical. It doesn’t come 100% naturally to me but it’s this skill I place real value in attaining. Throughout I referred frequently to the O’Rielly “Learning Pythonby Mark Lutz and David Ascher” book as I assume simply searching random Stackoverflow answers on fundamental concepts rather than edge cases can be contradictory. A couple quotes I found useful from the book were that “Functions are a nearly universal program-structuring device.” and that “Functions also provide a tool for splitting systems into pieces that have a well-defined role.”

I found the code for showing area notation which is cm\u00b2. I’ve completed the task (again after so many helpful corrections from the IDE!) but I feel there is a simpler way to write it/execute it. I made an initial attempt but it broke my code so I hope to improve in this area going forward. It feels like a really long set of if else variables. PyCharm was also helpful in suggesting I define the variable type as a string in my function. I tried to comment my code which is harder than I anticipated. I’ve read some various code on Github and the commenting has always been helpful but you never feel it is enough. But when you have spent so much time on a few lines or functions it seems so obvious it’s hard to see it through the eyes of the unfamiliar!

FIZZBUZZ

With regards to editing the FizzBuzz task what at first looked like a simple extending of the functionality with an extra parameter led me down a rabbit hole of re-doing everything and trying to find an optimal solution. It made me realise an initial mistake I had made in writing my function for the first task in understanding how the parameters would be overwritten. It helped to visualise working in a large codebase and how that might work! Secondly I struggled with the syntax of the second parameter to be evaluated. I’ve included my incorrect attempt as commented out code below my final solution.

Whilst reading up on the topic area I found this article helpful (https://www.digitalocean.com/community/tutorials/how-to-use-args-and-kwargs-in-python-3). I also found out about the kwargs function where you can create the possibility of entering more parameters than you perhaps had planned for. Perhaps we cover this flexibility next week.

While researching for a more optimal solution I discovered this (https://ditam.github.io/posts/fizzbuzz/) very interesting commentary on the challenge which appears to be a very common initial test in job applications. Although in JavaScript a lot of the concepts are fundamental and could be applied to Python. For me as a beginner the majority is way over my head and not possible to implement but I see it as a nice introduction to some terminology and the understanding of how complex code can really be. It will take me thousands of hours to reach a professional standard. In relation to this point there was a link to an apparently well known blog piece by the founder of Stackoverflow stating how many graduates fail this task of FizzBuzz! (https://blog.codinghorror.com/why-cant-programmers-program/). This is quite encouraging in a sense in highlighting how valuable the fundamentals are to attaining.

DATE TIME MODULE
import datetime
datetime.datetime.now().strftime("%Y-%m-%d-%H-%M-%S")

Running the above code produced the output:
2019-11-17-13-37-19
This matches with year, month, hours, minutes and seconds on my computer. I found from the docs that the method creates a formatted string from a given date, datetime or time object. Python strftime() > datetime object to string

SUMMARY

Although I’m frustrated at how long it’s taken me to complete the tasks satisfactorily, I also understand the necessity of getting the fundamentals right. I’ve experienced as a player and a teacher in my previous career how ‘awkward’ this initial skill acquisition can be – but it doesn’t make it feel any better! I just read rhis morning that I should at be least failing 15’5 of the time anyway! (https://www.independent.co.uk/news/science/failing-study-success-machine-learning-a9186051.html)

A difficult week time wise as I transition into a new job and struggle to write code in a short time frame! It’s very similar to a math problem, time just flies as you bang your head against the wall looking for the answer that you know exists! With a steep learning curve, extended tutorial activities and the responsibility to the studio project there is a pressure here which I do like. I’m guilty of procrastination by going and researching ‘off piste’ so this task list is teaching me to me more effective, really trying to Pareto’s principle into effect!

The lecture content is fantastic, must say I’m a fan of jumping into big new topics regularly. So many of the Youtube videos I have watched just seem to lack any real world scenario references. You could easily be left wondering what the point of a certain function is. Also learning by some of the multiple choice apps has been slow going. We are backing up everything with building a working application. I especially liked Raynes reference to a certain process where she mentioned the importance of it “when your 500 hundred lines down” in a codebase.

One of the key takeaways from the lecture were the value of abstraction, allowing for simpler processes to access/initiate complex use cases. Secondly there was a real emphasis on code re-useability – there is no point repeating yourself, but careful where you get code from!

We learnt about some of the standard Python library features, but again it was the implementation of them that really stuck with me. I look forward to getting to know some of these features much better! I’m beginning to see how much creativity there can be in coding, something I had read about it, but as you begin to see “how big the beast” really is, one can appreciate it more!

A further point was made on code efficiency when using libraries and functions. The analogy of only taking what you need as you would when carrying a heavy backpack is memorable. The dot notation allows us to call specific modules as we need them. Currently our project files are very small (some are only 1kb) but this will change quite soon apparently as we get more complex programmes working.



TUTORIAL NOTES

Our first task was very simple with use of the while function and a less than or equal to statement.

Our second task took me so much longer than it should have done for such few lines of code!! My initial plan turned out to be too ambitious to get it all working. I found some code snippets to implement validation for a number between 0 and 10 but I wasn’t able to get it working. This is clearly a frustrating failure on a standard element I will work further on. Also another mistake caused me to print endlessly the every increasing score as I has written the loop slightly wrong. Aside from very regular syntax errors, I’m happy to have a working version that fulfils the tutorials criteria. Using a helpful IDE improves my coding syntax with the instant feedback instead of spending time in a simple text editor.

Thirdly I coded up the alarm app. I feel like my final solution is readable and efficient for my stage of learning although lacking some of the error messages and input validation that would perhaps take it to the next level. One such attempt was a URL checker which I found on StackOverflow but I didn’t get it to work effectively.

The sleep function was interesting to learn about. Rather than simply a user experience feature, it actually has a purpose in multi threaded programming and can also help give the hardware time to catch up if there are high demands on the system.
Finally I thought I would look to find some Python code with a GUI aspect to the alarm clock. This (https://github.com/rajkumar49/Simple-Alarm-Clock-Python/blob/master/alarm-GUI.py) Github file was interesting to play with despite not understanding every line of the code.



UML
I probably would have been a lot faster working through these items by hand sketching but instead I wanted to have them on the computer so I used Canva to create them. I found this programme (https://yuml.me/diagram/scruffy/class/draw) used by some big names but felt it had too much of a learning curve.

I feel my solutions could have an extra layer of detail but I wanted to stay on track for the task. Coincidentally I’ve been doing something very similar in a UX task at my new job. It really is interesting how many use cases, requests and so on you could have as a user. For example the engineer working on the cash machine /ATM can probably think of a hundred separate problems he has to deal with whilst we view it very simply, very much in the abstract!

From reading the material and some pages from the O’Reilly book “Learning UML 2.0” I have noted the following.:
“To model a system, the most important aspect is to capture the dynamic behaviour. The main purpose of the use case diagram is to capture message flow from one object to another and to identify the requirements.
The activity diagrams show message flow from one activity to another, they include branching and parallel flow.
The grand purpose is to HELP not HINDER the system development and implementation process.“

I don’t have a professional level solution here but I noted I got faster and more organised with each diagram. The real value add here is that multiple non-technical people have a chance to contribute in the design process. Equally the clarity of thought this requires to do on a product ensures that there is reduced wasted time creating solutions with logic errors. This nearly goes as far as first principles thinking.

Flexible Variables & Programs

I’m repeating myself here but Rayne is a very clear speaker, fantastic pace and enunciation. It really adds value, in that concepts are easier memorised. Secondly these lectures have by far been the most engaging in that she codes or works in front of us. This live coding experience, the walkthrough experience is a type of learning I’ve enjoyed from sources such as Udemy and Youtube.

Data Types & Dynamic Typing

Dynamic typing and casting were terms I had seen thrown around on blogs and Twitter so it was nice to delve into what it really meant. The excerpts below from the official Python docs were of great help in what seems to be a trickier concept than it first appears. The reasoning laid out here also helps me understand why Python is so heavily recommended as a first language to learn. Reading up on this further shows that the helpfulness of the language in this aspect has a cost in the time it takes to execute the code.

“In a dynamically typed language, a variable is simply a value bound to a name; the value has a type — like “integer” or “string” or “list” — but the variable itself doesn’t. You could have a variable which, right now, holds a number, and later assign a string to it if you need it to change. “

“In a strongly typed language, you are simply not allowed to do anything that’s incompatible with the type of data you’re working with.” From the official Python docs (https://wiki.python.org/moin/Why%20is%20Python%20a%20dynamic%20language%20and%20also%20a%20strongly%20typed%20language)

The in built functions were presented to us so I intend to spend some time this week looking into popular use cases.
https://docs.python.org/3/library/functions.html

Rayne used the metaphor of a kitchen- dont need to know how its made/ what the specific ingredients are. Python lets you use very abstract functions – there is no need to allocate memory and define storage locations etc

“There may be times when you want to specify a type on to a variable. This can be done with casting. Python is an object-orientated language, and as such it uses classes to define data types, Casting in python is therefore done using constructor functions“ https://www.w3schools.com/python/python_casting.asp
I also found from Datacamp.com (an app I’ve used lately), that “in data science, you will often need to change the type of your data, so that it becomes easier to use and work with” and that casting is a prime way to do this. This led me to look at constructor functions in further detail. I found this most helpful :
“The name of a constructor is always the same, __init__(). The constructor can accept arguments when necessary to create the object. When you create a class without a constructor, Python automatically creates a default constructor for you that doesn’t do anything. Every class must have a constructor, even if it simply relies on the default constructor. ” (https://www.dummies.com/programming/python/how-to-create-a-constructor-in-python/). The complexity of programming is quite apparent as I got further off track and started reading about upcasting – I’ll leave that for another day!

I understood most of the table used in the lecture here but was curious about the last two terms. In turns out a hexadecimal string is a converter and returns TypeError when anything other than integer type constants are passed as parameters. An example is “The hexadecimal form of 23 is0x17”. oct(x) is similar and “ returns it’s octal representation in a string format.” Octal is the base number 8 system.

Casting can also be temporary or permanent. It is temporary if not assigned earlier in the function, if assigned in the print statement for example. If it is just assigned in the output code instead of after the = in the first piece of code. Type checking is something we will do in the future but its good practice to define inputs for security reasons to prevent ‘injection’ type attacks.

UML: I’ve started looking at some sections of the book “Learning UML 2.0“ By Kim Hamilton, Russell Miles from our recommended reading. I’ve recently started a new job and there is a lot of commonality in how logical processes have to be. Boolean logic is obviously the foundation of this. It strikes me already when working with others how much easier it is to deal in Boolean logic rather than vague descriptions of processes. Earlier coding experiences have made me familiar with the logic expressions and the indentation patterns. I found repetitive multiple choice tests helpful in making Boolean expressions seem intuitive.

*I’m aware at this point my notes are extensive. Perhaps its the self indulgence of writing in first person but I think it’s because as a teacher I’m aware of how important the initial building blocks are for skill acquisition.

TUTORIAL

1) I spent time trying to do this without reference and with a couple extra features and more conversational. Of course it was a bumpy process to end up at the screen below eventually. First of all I made multiple syntax errors such as missing a bracket, indentation, spelling error of a function, location of semi colons with else function. I also made logic errors initially, attempting multiple affirmative answers and writing two different outputs instead of using the one function, conversion. I tried to setup a backup answer/default but I couldn’t get the syntax correct. Look forward to improving upon this.

2) After a little bit of stack overflow and trial and error with the logic and assignment, I have something with three paths for either Fahrenheit, Celsius or a demo based on a non float/int answer. This could be simplified I’m sure but I wanted to write ‘long hand’ if you will for clear logic. I used the try function to test for errors in the initial question.

3) I have a working solution here but I need to spend more time going over the implimentation of true / false logic and also selecting multiple values. I commented out my attempt at checking the answer for other languages in a list but it just kept printing the whole list!

4) Another task that took way longer than expected but happy to complete it without incessant searching around the net! I know from previous study that this is very ineffecient code and could be refactored/ condensed somewhat but I guess I’ll to be patient at this stage.

What application would you most like to be able to create by the end of this course?

The end of this course is only a matter of weeks away so ambition appears in the next question…..It would be great to have a grasp of the fundamentals in all honesty. I own the O’Reilly book ‘Python for Data Analysis’ and to be able to make more sense of that, to run some of the code from it would be a good achievement. In my new role at work I would also hope to be able to contribute a little more on the code for our customer data. It’s very motivating for me not just to learn in the abstract sense, but to also see an effect in the real world.

What would you like to create in the future, once you’ve mastered Python?

First of all, I love the assumption that we will master Python! If I get that far I would like to think I have some degree of fluency in other languages too for reasons similar to being multi lingual in the natural languages. Opportunities, enjoyment and a greater understanding of the processes. Career wise I would love to achieve competency in machine learning applications and use not just the coding knowledge but the logic knowledge to become involved at the cutting edge of technology applications. Perhaps also to contribute to an open source project!

** I’ve contributed on the official Moodle discussions lightly. These forums have more potential for valuable interaction but there unused almost. Skype has bursts of value, but there is so much noise and excessive threads to catch up on always. I’ve really enjoyed discussing the course so far with a profficeint ‘R’ coder at my new job – fantastic insights about the awkward nature of learning to code!

LECTURE THOUGHTS

I watched the replay of the lecture later on as I had a busy day at work. We have definitely moved quickly on from printing hello world! Defining string types was something I made the obvious mistake on when doing some python tests online, defining a string instead of an integer. At work recently I’ve started working with charts (Highcharts specifically) and although its in JavaScript the idea is the same in that we play around with variables in the data. When building chatbots in Google Dialogflow I became aware of the need for the unicode standard due to certain emojis not displaying. Binary is a simple concept but very abstract in the unnatural way you count using it. It really is quite something to think that all of our technology comes from on / off gates! I knew Python had a reputation to be the easiest language to learn and that it was very high level without really knowing what was meant.

Today made it crystal clear that this is because Python makes a lot of decisions on your behalf such as memory allocation and also works flexibly allowing for dynamic variables. It enables a concept similar to over-writing during assignment. Again the usage of Dialogflow served as a course in myCamelCase naming patterns. Also at work, our shared folder has so much information we need a common naming schema. The intro to mathematic operators was similar to the Udemy course but now with a data science focus I see how common our need will be for this in the future. As a final point, reading the Google style guide makes you realise how deep this subject is. Is it even possible to know the majority without dedicating your life to it?

TUTORIAL

1 – max value -1 = b^n – 1
a.7 bits = 1111111 = 2^7 – 1 = 127
b.16 bits = 2^16-1 = 65535
c.32 bits = 2^16-1 = 4294967295
d.64 bits= 2^64 -1 =18446744073709551615

2 – Convert the following binary numbers into decimal:
a.00001100 = 2^2+2^3=12
b.01110001 = 2^0+2^4+2^5,+2^6= 1 + 16 + 32 +64 = 113
c.01100011 = 2^0 + 2^1 + 2^5 + 2^6 = 1+2+32+64 = 99
d.10011011 = 2^0 + 2^1 + 2^3 + 2^4 +2^7 = 1+2+8+16+128=155

3. Determine which ASCII character is represented by each of the following:
a.01001101: Uppercase M
b.00111101: Equals
c.00000000: Null char
d.10010001: Left single quotation mark

4.Floating point numbers are thus named because there is no defined number of numbers either side of the decimal point.


PRACTICAL
I made a few syntax errors before being able to do it without reference, let’s just say I shall never forget a comma again! I had two big points of learning. Number one I took a lot of time trying to find ways to align the output. After a while I got help from my studio partner and used the tab syntax, ‘\t’ although this doesn’t feel very optimised and certainly not automated for large scale outputs. Below are some images of other discoveries about ways to align. Especially when it comes to API’s and outputting in readable format, I think this will be a popular area to optimise.



The second issue was accepting user input as a number. Without using the int function the string type is wrong. Took me a little while to figure this out but I know this ‘pain’ of not doing something straight away is what helps internalise something, what helps learn something. I know this from my previous domain first as a competitor and latterly as a teacher.



ADDITIONAL ACTIVITIES
“The sys module is a set of functions which provide crucial information about how your Python script is interacting with the host system . In Python, the import statement allows you to make use of data and functionality from a module which isn’t the current one.“

This imports the module.
The second code shows you your system type. I also discovered this related function: (src: https://www.webucator.com/how-to/how-check-the-operating-system-with-python.cfm)

2**100 means carrying out math – 2 to the power of 100!
This assigns the string value to x.
This prints x’s value 8 times.

I used the above function to complete the final task, below.

TUTORIAL

REFLECTIONS DAY 2

A really nicely presented lecture, eloquent and informative. As a ‘#codenewbie there is a nice feeling to running a small little program in the editor but this is something I’ve done many times now. It was today’s extra activity in the tutorial which really caught my attention as it tests and demands real logic. The clear need for efficiency, for minimal code, is something I’ve read about being highly valued in industry. Knowing this I’m sure there is a far better way to code the earlier small program when allowing for potential extra people. It occurs to me that despite an enjoyment of math I really haven’t worked with algorithms. I hope one day to be able to say I’m proficient in this area.

I liked that the extra reading provided (similar to earlier in the semester) provided historical context to our studies. It’s easy to feel your engaged in an ultra modern activity, sitting on the internet on a powerful machine, but in essence you are often engaging in pure logic, a timeless activity. One of the reasons to desire to code to a high standard is that it means you will have successfully ‘stood on the shoulders of giants’. I mean to say you have applied the lessons from history to deliver efficiency and quality.

I’m mindful also that experienced coders ‘hate the mouse’ and rely heavily on keyboard shortcuts. With that in mind, here are my two most used in these early days: To save press ctrl and s. To run the interpreter use ctrl, shft and f10.

Kordemsky Puzzle

After first failing to keep all the pieces positions in my head, I opened up Trello to track movements. On the right I had a reference for positions and on the left I moved the pieces around whilst taking note of moves so as to look for patterns. The amount of time I’ve used here is impractical for the actual case but I feel I gained something from a totally different approach other than brute force in such a new area for me. I also wanted to consider the problem as something that could be applied on a much larger scale. eg if there was 101 spaces and 100 pieces.

After some tries I decided to codify my thinking towards the problem as best I could rather than simply log the moves I made. I achieved 16 moves. It really bugged me that I didn’t complete the task so I plan to return over the weekend. Here are the notes so far.

Algorithm goal: swap position of pieces on board to opposite sides.
Rule : if all x pieces are left of the space and all y pieces are right of the space at start, the opposite should be true to complete task.

Leave a Reply

Your email address will not be published. Required fields are marked *