def entropy(keystream, base=None): vc = pd.Series(keystream).value_counts(normalize=True, sort=False) base = e if base is None else base #return -(vc * np.log(vc)/np.log(base)).sum() entropy = -(vc * np.log(vc)/np.log(base)).sum() print("entropy = ", entropy)

# Entropy calculator

Page 1 of 1## 4 Replies - 369 Views - Last Post: 02 September 2019 - 04:54 PM

### #1

# Entropy calculator

Posted 02 September 2019 - 06:57 AM

I want to calculate the entropy of a set of numbers. These numbers are in a list, keystream. When I run the code below I get a number between 2 and 4.85. How can I get the entropy to be scored between 0 and 1? What do these numbers above 1 mean exactly? I get that the higher they are the more unpredictable the outcomes. Thanks.

##
**Replies To:** Entropy calculator

### #2

## Re: Entropy calculator

Posted 02 September 2019 - 10:18 AM

Are you taking the log of the each probability? Recall that entropy takes the sum of the terms of the form:

-Pr[X = i] * log(Pr[X = i]).

The negative sign accounts for the fact that log(Pr[X = i]) is negative.

Note that we omit terms where Pr[X = i] = 0.

-Pr[X = i] * log(Pr[X = i]).

The negative sign accounts for the fact that log(Pr[X = i]) is negative.

Note that we omit terms where Pr[X = i] = 0.

### #3

## Re: Entropy calculator

Posted 02 September 2019 - 02:24 PM

Not exactly sure. I found this code online as well. I was assuming this part of the code took the probability and stored it in norm_counts.

But this gives me a number above 1 as well.

Sorry pasted the wrong code earlier.

def entropy2(keystream, base=None): value,counts = np.unique(keystream, return_counts=True) norm_counts = counts / counts.sum() base = e if base is None else base #return -(norm_counts * np.log(norm_counts)/np.log(base)).sum() entropy = -(norm_counts * np.log(norm_counts)/np.log(base)).sum() print("entropy2 = ", entropy)

But this gives me a number above 1 as well.

Sorry pasted the wrong code earlier.

This post has been edited by **2Dabz**: 02 September 2019 - 02:50 PM

### #4

## Re: Entropy calculator

Posted 02 September 2019 - 02:39 PM

The "keystream" is a list of numbers representing the possible 256 ASCII characters. Some runs return sets of 128 characters. In some runs the 128 characters have a perfectly even distribution. This run of 500 produced these numbers at these frequencies. I am deriving them from a PRNG similar to Blum Blum Shub. They have shorter periods but are supposed to be more secure and unpredictable than PRNGs with higher periods. So this is the type of data I am trying to analyze.

(165, 40), (231, 40), (113, 40), (91, 40), (173, 40), (63, 40), (25, 40), (83, 40), (117, 39), (87, 39), (129, 39), (11, 39), (253, 39), (47, 39), (169, 39), (131, 39), (69, 39), (199, 39), (145, 39), (187, 39), (77, 39), (31, 39), (57, 39), (179, 39), (21, 39), (55, 39), (161, 39), (107, 39), (157, 39), (15, 39), (201, 39), (227, 39), (229, 39), (167, 39), (177, 39), (27, 39), (237, 39), (255, 39), (89, 39), (19, 39), (181, 39), (23, 39), (193, 39), (203, 39), (61, 39), (239, 39), (233, 39), (67, 39), (133, 39), (135, 39), (209, 39), (123, 39), (141, 39), (223, 39), (121, 39), (115, 39), (85, 39), (247, 39), (225, 39), (43, 39), (221, 39), (207, 39), (9, 39), (163, 39), (37, 39), (103, 39), (241, 39), (219, 39), (45, 39), (191, 39), (153, 39), (211, 39), (245, 39), (215, 39), (1, 39), (139, 39), (125, 39), (175, 39), (41, 39), (3, 39), (197, 39), (71, 39), (17, 39), (59, 39), (205, 39), (159, 39), (185, 39), (51, 39), (149, 39), (183, 39), (33, 39), (235, 39), (29, 39), (143, 39), (73, 39), (99, 39), (101, 39), (39, 39), (49, 39), (155, 39), (109, 39), (127, 39), (217, 39), (147, 39), (53, 39), (151, 39), (65, 39), (75, 39), (189, 39), (111, 39), (105, 39), (195, 39), (5, 39), (7, 39), (81, 39), (251, 39), (13, 39), (95, 39), (249, 39), (243, 39), (213, 39), (119, 39), (97, 39), (171, 39), (93, 39), (79, 39), (137, 39), (35, 39)

(165, 40), (231, 40), (113, 40), (91, 40), (173, 40), (63, 40), (25, 40), (83, 40), (117, 39), (87, 39), (129, 39), (11, 39), (253, 39), (47, 39), (169, 39), (131, 39), (69, 39), (199, 39), (145, 39), (187, 39), (77, 39), (31, 39), (57, 39), (179, 39), (21, 39), (55, 39), (161, 39), (107, 39), (157, 39), (15, 39), (201, 39), (227, 39), (229, 39), (167, 39), (177, 39), (27, 39), (237, 39), (255, 39), (89, 39), (19, 39), (181, 39), (23, 39), (193, 39), (203, 39), (61, 39), (239, 39), (233, 39), (67, 39), (133, 39), (135, 39), (209, 39), (123, 39), (141, 39), (223, 39), (121, 39), (115, 39), (85, 39), (247, 39), (225, 39), (43, 39), (221, 39), (207, 39), (9, 39), (163, 39), (37, 39), (103, 39), (241, 39), (219, 39), (45, 39), (191, 39), (153, 39), (211, 39), (245, 39), (215, 39), (1, 39), (139, 39), (125, 39), (175, 39), (41, 39), (3, 39), (197, 39), (71, 39), (17, 39), (59, 39), (205, 39), (159, 39), (185, 39), (51, 39), (149, 39), (183, 39), (33, 39), (235, 39), (29, 39), (143, 39), (73, 39), (99, 39), (101, 39), (39, 39), (49, 39), (155, 39), (109, 39), (127, 39), (217, 39), (147, 39), (53, 39), (151, 39), (65, 39), (75, 39), (189, 39), (111, 39), (105, 39), (195, 39), (5, 39), (7, 39), (81, 39), (251, 39), (13, 39), (95, 39), (249, 39), (243, 39), (213, 39), (119, 39), (97, 39), (171, 39), (93, 39), (79, 39), (137, 39), (35, 39)

This post has been edited by **2Dabz**: 02 September 2019 - 02:40 PM

### #5

## Re: Entropy calculator

Posted 02 September 2019 - 04:54 PM

Found this and think it makes sense. I am getting 4.85 for a range of 128 or out of a possible value of 7.

>>> H(range(0,256))

8.0

>>> H(range(0,64))

6.0

>>> H(range(0,128))

7.0

>>> H([0,1])

1.0

>>> H('Standard English text usually falls somewhere between 3.5 and 5')

4.228788210509104

>>> H(range(0,256))

8.0

>>> H(range(0,64))

6.0

>>> H(range(0,128))

7.0

>>> H([0,1])

1.0

>>> H('Standard English text usually falls somewhere between 3.5 and 5')

4.228788210509104

Page 1 of 1