4 Replies - 401 Views - Last Post: 02 September 2019 - 04:54 PM Rate Topic: -----

#1 2Dabz   User is offline

  • D.I.C Head

Reputation: 2
  • View blog
  • Posts: 95
  • Joined: 10-September 17

Entropy calculator

Posted 02 September 2019 - 06:57 AM

I want to calculate the entropy of a set of numbers. These numbers are in a list, keystream. When I run the code below I get a number between 2 and 4.85. How can I get the entropy to be scored between 0 and 1? What do these numbers above 1 mean exactly? I get that the higher they are the more unpredictable the outcomes. Thanks.

def entropy(keystream, base=None):
    vc = pd.Series(keystream).value_counts(normalize=True, sort=False)
    base = e if base is None else base
    #return -(vc * np.log(vc)/np.log(base)).sum()
    entropy = -(vc * np.log(vc)/np.log(base)).sum()
    print("entropy = ", entropy)



Is This A Good Question/Topic? 0
  • +

Replies To: Entropy calculator

#2 macosxnerd101   User is online

  • Games, Graphs, and Auctions
  • member icon




Reputation: 12646
  • View blog
  • Posts: 45,820
  • Joined: 27-December 08

Re: Entropy calculator

Posted 02 September 2019 - 10:18 AM

Are you taking the log of the each probability? Recall that entropy takes the sum of the terms of the form:

-Pr[X = i] * log(Pr[X = i]).

The negative sign accounts for the fact that log(Pr[X = i]) is negative.

Note that we omit terms where Pr[X = i] = 0.
Was This Post Helpful? 0
  • +
  • -

#3 2Dabz   User is offline

  • D.I.C Head

Reputation: 2
  • View blog
  • Posts: 95
  • Joined: 10-September 17

Re: Entropy calculator

Posted 02 September 2019 - 02:24 PM

Not exactly sure. I found this code online as well. I was assuming this part of the code took the probability and stored it in norm_counts.
def entropy2(keystream, base=None):
    value,counts = np.unique(keystream, return_counts=True)
    norm_counts = counts / counts.sum()
    base = e if base is None else base
    #return -(norm_counts * np.log(norm_counts)/np.log(base)).sum()
    entropy = -(norm_counts * np.log(norm_counts)/np.log(base)).sum()
    print("entropy2 = ", entropy)



But this gives me a number above 1 as well.

Sorry pasted the wrong code earlier.

This post has been edited by 2Dabz: 02 September 2019 - 02:50 PM

Was This Post Helpful? 0
  • +
  • -

#4 2Dabz   User is offline

  • D.I.C Head

Reputation: 2
  • View blog
  • Posts: 95
  • Joined: 10-September 17

Re: Entropy calculator

Posted 02 September 2019 - 02:39 PM

The "keystream" is a list of numbers representing the possible 256 ASCII characters. Some runs return sets of 128 characters. In some runs the 128 characters have a perfectly even distribution. This run of 500 produced these numbers at these frequencies. I am deriving them from a PRNG similar to Blum Blum Shub. They have shorter periods but are supposed to be more secure and unpredictable than PRNGs with higher periods. So this is the type of data I am trying to analyze.

(165, 40), (231, 40), (113, 40), (91, 40), (173, 40), (63, 40), (25, 40), (83, 40), (117, 39), (87, 39), (129, 39), (11, 39), (253, 39), (47, 39), (169, 39), (131, 39), (69, 39), (199, 39), (145, 39), (187, 39), (77, 39), (31, 39), (57, 39), (179, 39), (21, 39), (55, 39), (161, 39), (107, 39), (157, 39), (15, 39), (201, 39), (227, 39), (229, 39), (167, 39), (177, 39), (27, 39), (237, 39), (255, 39), (89, 39), (19, 39), (181, 39), (23, 39), (193, 39), (203, 39), (61, 39), (239, 39), (233, 39), (67, 39), (133, 39), (135, 39), (209, 39), (123, 39), (141, 39), (223, 39), (121, 39), (115, 39), (85, 39), (247, 39), (225, 39), (43, 39), (221, 39), (207, 39), (9, 39), (163, 39), (37, 39), (103, 39), (241, 39), (219, 39), (45, 39), (191, 39), (153, 39), (211, 39), (245, 39), (215, 39), (1, 39), (139, 39), (125, 39), (175, 39), (41, 39), (3, 39), (197, 39), (71, 39), (17, 39), (59, 39), (205, 39), (159, 39), (185, 39), (51, 39), (149, 39), (183, 39), (33, 39), (235, 39), (29, 39), (143, 39), (73, 39), (99, 39), (101, 39), (39, 39), (49, 39), (155, 39), (109, 39), (127, 39), (217, 39), (147, 39), (53, 39), (151, 39), (65, 39), (75, 39), (189, 39), (111, 39), (105, 39), (195, 39), (5, 39), (7, 39), (81, 39), (251, 39), (13, 39), (95, 39), (249, 39), (243, 39), (213, 39), (119, 39), (97, 39), (171, 39), (93, 39), (79, 39), (137, 39), (35, 39)

This post has been edited by 2Dabz: 02 September 2019 - 02:40 PM

Was This Post Helpful? 0
  • +
  • -

#5 2Dabz   User is offline

  • D.I.C Head

Reputation: 2
  • View blog
  • Posts: 95
  • Joined: 10-September 17

Re: Entropy calculator

Posted 02 September 2019 - 04:54 PM

Found this and think it makes sense. I am getting 4.85 for a range of 128 or out of a possible value of 7.

>>> H(range(0,256))
8.0
>>> H(range(0,64))
6.0
>>> H(range(0,128))
7.0
>>> H([0,1])
1.0
>>> H('Standard English text usually falls somewhere between 3.5 and 5')
4.228788210509104
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1