# Entropy calculator

Page 1 of 1

## 4 Replies - 401 Views - Last Post: 02 September 2019 - 04:54 PMRate Topic: //<![CDATA[ rating = new ipb.rating( 'topic_rate_', { url: 'https://www.dreamincode.net/forums/index.php?app=forums&module=ajax&section=topics&do=rateTopic&t=417211&amp;s=adf555573c09b32a98c67fcaeff49d70&md5check=' + ipb.vars['secure_hash'], cur_rating: 0, rated: 0, allow_rate: 0, multi_rate: 1, show_rate_text: true } ); //]]>

### #1 2Dabz

Reputation: 2
• Posts: 95
• Joined: 10-September 17

# Entropy calculator

Posted 02 September 2019 - 06:57 AM

I want to calculate the entropy of a set of numbers. These numbers are in a list, keystream. When I run the code below I get a number between 2 and 4.85. How can I get the entropy to be scored between 0 and 1? What do these numbers above 1 mean exactly? I get that the higher they are the more unpredictable the outcomes. Thanks.

```def entropy(keystream, base=None):
vc = pd.Series(keystream).value_counts(normalize=True, sort=False)
base = e if base is None else base
#return -(vc * np.log(vc)/np.log(base)).sum()
entropy = -(vc * np.log(vc)/np.log(base)).sum()
print("entropy = ", entropy)

```

Is This A Good Question/Topic? 0

## Replies To: Entropy calculator

### #2 macosxnerd101

• Games, Graphs, and Auctions

Reputation: 12646
• Posts: 45,820
• Joined: 27-December 08

## Re: Entropy calculator

Posted 02 September 2019 - 10:18 AM

Are you taking the log of the each probability? Recall that entropy takes the sum of the terms of the form:

-Pr[X = i] * log(Pr[X = i]).

The negative sign accounts for the fact that log(Pr[X = i]) is negative.

Note that we omit terms where Pr[X = i] = 0.

### #3 2Dabz

Reputation: 2
• Posts: 95
• Joined: 10-September 17

## Re: Entropy calculator

Posted 02 September 2019 - 02:24 PM

Not exactly sure. I found this code online as well. I was assuming this part of the code took the probability and stored it in norm_counts.
```def entropy2(keystream, base=None):
value,counts = np.unique(keystream, return_counts=True)
norm_counts = counts / counts.sum()
base = e if base is None else base
#return -(norm_counts * np.log(norm_counts)/np.log(base)).sum()
entropy = -(norm_counts * np.log(norm_counts)/np.log(base)).sum()
print("entropy2 = ", entropy)

```

But this gives me a number above 1 as well.

Sorry pasted the wrong code earlier.

This post has been edited by 2Dabz: 02 September 2019 - 02:50 PM

### #4 2Dabz

Reputation: 2
• Posts: 95
• Joined: 10-September 17

## Re: Entropy calculator

Posted 02 September 2019 - 02:39 PM

The "keystream" is a list of numbers representing the possible 256 ASCII characters. Some runs return sets of 128 characters. In some runs the 128 characters have a perfectly even distribution. This run of 500 produced these numbers at these frequencies. I am deriving them from a PRNG similar to Blum Blum Shub. They have shorter periods but are supposed to be more secure and unpredictable than PRNGs with higher periods. So this is the type of data I am trying to analyze.

(165, 40), (231, 40), (113, 40), (91, 40), (173, 40), (63, 40), (25, 40), (83, 40), (117, 39), (87, 39), (129, 39), (11, 39), (253, 39), (47, 39), (169, 39), (131, 39), (69, 39), (199, 39), (145, 39), (187, 39), (77, 39), (31, 39), (57, 39), (179, 39), (21, 39), (55, 39), (161, 39), (107, 39), (157, 39), (15, 39), (201, 39), (227, 39), (229, 39), (167, 39), (177, 39), (27, 39), (237, 39), (255, 39), (89, 39), (19, 39), (181, 39), (23, 39), (193, 39), (203, 39), (61, 39), (239, 39), (233, 39), (67, 39), (133, 39), (135, 39), (209, 39), (123, 39), (141, 39), (223, 39), (121, 39), (115, 39), (85, 39), (247, 39), (225, 39), (43, 39), (221, 39), (207, 39), (9, 39), (163, 39), (37, 39), (103, 39), (241, 39), (219, 39), (45, 39), (191, 39), (153, 39), (211, 39), (245, 39), (215, 39), (1, 39), (139, 39), (125, 39), (175, 39), (41, 39), (3, 39), (197, 39), (71, 39), (17, 39), (59, 39), (205, 39), (159, 39), (185, 39), (51, 39), (149, 39), (183, 39), (33, 39), (235, 39), (29, 39), (143, 39), (73, 39), (99, 39), (101, 39), (39, 39), (49, 39), (155, 39), (109, 39), (127, 39), (217, 39), (147, 39), (53, 39), (151, 39), (65, 39), (75, 39), (189, 39), (111, 39), (105, 39), (195, 39), (5, 39), (7, 39), (81, 39), (251, 39), (13, 39), (95, 39), (249, 39), (243, 39), (213, 39), (119, 39), (97, 39), (171, 39), (93, 39), (79, 39), (137, 39), (35, 39)

This post has been edited by 2Dabz: 02 September 2019 - 02:40 PM

### #5 2Dabz

Reputation: 2
• Posts: 95
• Joined: 10-September 17

## Re: Entropy calculator

Posted 02 September 2019 - 04:54 PM

Found this and think it makes sense. I am getting 4.85 for a range of 128 or out of a possible value of 7.

>>> H(range(0,256))
8.0
>>> H(range(0,64))
6.0
>>> H(range(0,128))
7.0
>>> H([0,1])
1.0
>>> H('Standard English text usually falls somewhere between 3.5 and 5')
4.228788210509104