The Downside
This is a considerably contrivied instance to show an issue. I have to create a string with the phrase phrase 100,000 occasions. What is the quickest approach to generate this string? I might use string concatenation (merely + the strings to at least one one other). I might be a part of an inventory with 100,000 objects.
My proposed code for this instance is under
def concat_string(phrase: str, iterations: int = 100000) -> str:
final_string = ""
for i in vary(iterations):
final_string += phrase
return final_string
def join_string(phrase: str, iterations: int = 100000) -> str:
final_string = []
for i in vary(iterations):
final_string.append(phrase)
return "".be a part of(final_string)
Outcomes
In concat_string, I iterate 100,000 objects with every iteration including my phrase to the tip of the string. In join_string, I append my phrase to an inventory on every iteration after which be a part of the listing right into a string on the finish.
Working every operate by the inbuilt profiler (cProfile) reveals how these two capabilities carry out.
>>> cProfile.run('concat_string("phrase ")')
4 operate calls in 1.026 seconds
Ordered by: commonplace identify
ncalls tottime percall cumtime percall filename:lineno(operate)
1 0.000 0.000 1.026 1.026 :1()
1 1.026 1.026 1.026 1.026 take a look at.py:9(concat_string)
1 0.000 0.000 1.026 1.026 {built-in methodology builtins.exec}
1 0.000 0.000 0.000 0.000 {methodology 'disable' of '_lsprof.Profiler' objects}
>>> cProfile.run('join_string("phrase ")')
100005 operate calls in 0.013 seconds
Ordered by: commonplace identify
ncalls tottime percall cumtime percall filename:lineno(operate)
1 0.000 0.000 0.013 0.013 :1()
1 0.009 0.009 0.013 0.013 take a look at.py:16(join_string)
1 0.000 0.000 0.013 0.013 {built-in methodology builtins.exec}
100000 0.004 0.000 0.004 0.000 {methodology 'append' of 'listing' objects}
1 0.000 0.000 0.000 0.000 {methodology 'disable' of '_lsprof.Profiler' objects}
1 0.000 0.000 0.000 0.000 {methodology 'be a part of' of 'str' objects}
What’s occurring right here?
be a part of is greater than 75x than the concatination methodology. Why?
String are immutable objects in Python. I talked about these in my final Gotcha Article about default parameters. This immutability implies that a string cannot be modified. concat_string does look like altering the string with every + motion, however beneath the hood, Python has to create a brand new string object every iteration by the loop. Which means there are 99,999 non permanent string values – creating and discarding nearly all of them instantly on the following iteration throughout the concatenation motion.
join_string alternatively, is appending 100,000 string objects to an inventory. However, just one listing is created. The ultimate be a part of is barely doing a single concatenation with all 100,000 strings.
What are the implications of this?
Whereas it is a contrived instance to point out the issue, there are actual efficiency impacts to string immutability that might not be apparent. There are different locations the place a brand new string is created generally utilized in Python. A pair examples are f-strings, %s format specifiers and .format(). Every of those create a model new string.
This does not imply you need to keep away from these, because the efficiency impression is barely actually apparent in conditions the place you’re appending loads of strings collectively. Nevertheless, when you’ve got a string formatting line in a loop, it is a potential space to give attention to for efficiency enhancements.
