School Assignment? Project Due Tomorrow? Chat LIVE With A Programming Expert!

Welcome to Dream.In.Code
Become an Expert!

Join 307,124 Programmers for FREE! Get instant access to thousands of experts, tutorials, code snippets, and more! There are 2,039 people online right now. Registration is fast and FREE... Join Now!




Data mining application

 

Data mining application, how to start

alowais

13 Sep, 2009 - 07:15 PM
Post #1

New D.I.C Head
*

Joined: 13 Sep, 2008
Posts: 45

Hi everyone ,

I'm about to implement an application using data mining technique but i really don't know how to start designing it.

I want suggestion and ideas from you.

The application is to predict the good credit risk from bad credit risk customers, when a customer wants to obtain a credit card or a loan from the bank ,they will ask for information which normally will be the income ,age ,citizenship ..etc..

So the bank based on the information provided will decide on weather this applicant has a high risk or low risk weather he will pay the loan or the credit card payments or not.


Thanksssz,







User is offlineProfile CardPM
+Quote Post


modi123_1

RE: Data Mining Application

14 Sep, 2009 - 07:57 AM
Post #2

Suiter #2
Group Icon

Joined: 12 Jun, 2008
Posts: 1,837



Thanked: 80 times
Dream Kudos: 150
My Contributions
Ah.. yeah.. if you are asking the question it might be out of your league. I mean that is an insane amount of data to make this functional for a classroom let alone running a business off it.

http://www.theiia.org/intAuditor/itaudit/a...and-techniques/

This post has been edited by modi123_1: 14 Sep, 2009 - 07:59 AM
User is online!Profile CardPM
+Quote Post

KYA

RE: Data Mining Application

14 Sep, 2009 - 07:59 AM
Post #3

#include <nerd.h>
Group Icon

Joined: 14 Sep, 2007
Posts: 11,500



Thanked: 508 times
Dream Kudos: 2875
Expert In: C, C++, Java

My Contributions
Hmmmm, if the credit score (FICO) is already calculated, do a query on anything > #theBankIsWillingToTolerate

[camel case ftw]

If you need to actually compute a credit score to determine risk, then the scope of the project just got a lot bigger. A lot.
User is online!Profile CardPM
+Quote Post

MajorWalrus

RE: Data Mining Application

14 Sep, 2009 - 10:13 AM
Post #4

D.I.C Head
**

Joined: 22 Apr, 2009
Posts: 98



Thanked: 9 times
My Contributions
Yeah. Seems like you not doing mining so much. Cause the customer is going to provide all the required information. Right?

So its basically an algorithm that computes risk based on some formula. The bank would have to determine criteria and weights for the various variables.

Also, weather = rain, clouds, ect. You mean "whether." smile.gif

Good luck!
User is offlineProfile CardPM
+Quote Post

alowais

RE: Data Mining Application

14 Sep, 2009 - 11:04 PM
Post #5

New D.I.C Head
*

Joined: 13 Sep, 2008
Posts: 45

loolzz..
Ok Mr.weather ,you're right the data will be provided but how should I determine criteria and weights for the various variables ?
I mean based on What the decision will be taken to accept or reject the customer?

User is offlineProfile CardPM
+Quote Post

modi123_1

RE: Data Mining Application

15 Sep, 2009 - 06:48 AM
Post #6

Suiter #2
Group Icon

Joined: 12 Jun, 2008
Posts: 1,837



Thanked: 80 times
Dream Kudos: 150
My Contributions
QUOTE(alowais @ 15 Sep, 2009 - 01:04 AM) *

loolzz..
Ok Mr.weather ,you're right the data will be provided but how should I determine criteria and weights for the various variables ?
I mean based on What the decision will be taken to accept or reject the customer?


Mr. Weather? Huh?

The criteria and weight are determined by you, your bosses, or who ever has a stake in the output. Based on the output work backwards.
User is online!Profile CardPM
+Quote Post

KYA

RE: Data Mining Application

15 Sep, 2009 - 07:37 PM
Post #7

#include <nerd.h>
Group Icon

Joined: 14 Sep, 2007
Posts: 11,500



Thanked: 508 times
Dream Kudos: 2875
Expert In: C, C++, Java

My Contributions
I would say the bank. They have the final authority on who gets a loan. Ask them for the criteria they would normally review "by hand" and then devise an algorithm based on the information. You've then automated the process (which is their goal).

This post has been edited by KYA: 15 Sep, 2009 - 07:37 PM
User is online!Profile CardPM
+Quote Post

no2pencil

RE: Data Mining Application

15 Sep, 2009 - 07:39 PM
Post #8

i R L33t Skiddie, k?
Group Icon

Joined: 10 May, 2007
Posts: 13,494



Thanked: 303 times
Dream Kudos: 2875
Expert In: Goofing Off

My Contributions
Where are you planning on mining this data & what are you planning to find?
User is online!Profile CardPM
+Quote Post

alowais

RE: Data Mining Application

16 Sep, 2009 - 05:43 AM
Post #9

New D.I.C Head
*

Joined: 13 Sep, 2008
Posts: 45

QUOTE(KYA @ 15 Sep, 2009 - 07:37 PM) *

I would say the bank. They have the final authority on who gets a loan. Ask them for the criteria they would normally review "by hand" and then devise an algorithm based on the information. You've then automated the process (which is their goal).


OK let us say they asked the customer about the age , education ,citizenship ,material status ,income and gender , they will consider the higher age and income case better than the lower ones and the citizenship is also an imporatant factor.

So the question here ist just that simple? understand the relation between those attributes and then finsh? (No way!!)


QUOTE(no2pencil @ 15 Sep, 2009 - 07:39 PM) *

Where are you planning on mining this data & what are you planning to find?


What am planning to find ? Is the decision to finance the customer or provide him with a credit card.

But "Where are you planning on mining this data" I don't understand what you meant by where.

Thanks,
User is offlineProfile CardPM
+Quote Post

modi123_1

RE: Data Mining Application

16 Sep, 2009 - 08:33 AM
Post #10

Suiter #2
Group Icon

Joined: 12 Jun, 2008
Posts: 1,837



Thanked: 80 times
Dream Kudos: 150
My Contributions
QUOTE(alowais @ 16 Sep, 2009 - 07:43 AM) *

QUOTE(KYA @ 15 Sep, 2009 - 07:37 PM) *

I would say the bank. They have the final authority on who gets a loan. Ask them for the criteria they would normally review "by hand" and then devise an algorithm based on the information. You've then automated the process (which is their goal).


OK let us say they asked the customer about the age , education ,citizenship ,material status ,income and gender , they will consider the higher age and income case better than the lower ones and the citizenship is also an imporatant factor.

So the question here ist just that simple? understand the relation between those attributes and then finsh? (No way!!)


QUOTE(no2pencil @ 15 Sep, 2009 - 07:39 PM) *

Where are you planning on mining this data & what are you planning to find?


What am planning to find ? Is the decision to finance the customer or provide him with a credit card.

But "Where are you planning on mining this data" I don't understand what you meant by where.

Thanks,


Where as in what industry? Is this for fun? For a job? For school?

Again it's not your job to come up with the relations of data - but your job to implement it.
User is online!Profile CardPM
+Quote Post

alowais

RE: Data Mining Application

16 Sep, 2009 - 09:15 AM
Post #11

New D.I.C Head
*

Joined: 13 Sep, 2008
Posts: 45

It's a project I've been asked to do.
User is offlineProfile CardPM
+Quote Post

Aeternalis

RE: Data Mining Application

16 Sep, 2009 - 12:39 PM
Post #12

D.I.C Regular
***

Joined: 13 Jul, 2009
Posts: 273



Thanked: 25 times
My Contributions
Credit card companies pay big bucks to data mining specialists to recover this type of information from their databases. The information they use to determine a persons credit worthiness is also recalculated often to determine if they should lower/ raise their limit or close the account, or other decisions that are made to mitigate their risk exposure.

The actual formulas they use to calculate the risk for each person is based on statistical data that is mined from their databases and is considered proprietary information. They keep it secret for good reason as their competitors could use that information to increase their own market position. by undercutting them, or outselling them. Generally the credit score is not determined by aggregating these factors, but instead the credit score is one factor of many that help them determine your risk.

Since your work is just for a school project, you wont have to concern yourself with actual implementation scenarios. Just create your own proprietary formulas that are logical and are based off of the data you are provided. As long as your getting your information from the data provided and using that information to make decisions about what risks to take on which customers, you should be in the clear. After all data mining is just about turning data into useable information.

Aet


User is offlineProfile CardPM
+Quote Post

Jayman

RE: Data Mining Application

16 Sep, 2009 - 01:07 PM
Post #13

Student of Life
Group Icon

Joined: 26 Dec, 2005
Posts: 8,570



Thanked: 237 times
Dream Kudos: 500
Expert In: Everything

My Contributions
You should be asking these questions from whomever assigned you the project. They are the ones who will define the criteria that will determine the amount of risk.

There is not a specific formula to calculate risk, the formula is specific to the clients needs.
User is offlineProfile CardPM
+Quote Post

NickDMax

RE: Data Mining Application

16 Sep, 2009 - 01:57 PM
Post #14

Can grep dead trees!
Group Icon

Joined: 18 Feb, 2007
Posts: 5,267



Thanked: 293 times
Dream Kudos: 1175
Expert In: Java/C++

My Contributions
The original post does not really seem to describe data mining to me. Data mining is about finding knowledge. It looks for patterns and correlations that are not readily apparent from the data.

Data mining would not be much help in determining if user A is a good risk or a bad risk -- it would be helpful in formulating rules for telling if people who meet certain criteria are a good risk or a bad risk.

Data mining is done with large data-sets and generally takes a good amount of time to run and even longer to analyze the data to sort out noise from quality knowledge. So you don't re-mine every time a user requests a credit card! That would not be cost effective and really would not generate any new information (on the small scale, it should generally take quite a few new data points to represent a new pattern).

So what you are asking about is not data-mining but what people do with the knowledge gained. i.e -- Using data mining to form statistical profiles used for risk assessments.

The problem with your idea is getting the data to initially mine. You would need 1000's of credit histories along with the correlating personal meta-data. While large credit institutions would have such data -- there are very strict rules upon how they must treat that data and what they are allowed to use it for.
User is offlineProfile CardPM
+Quote Post

alowais

RE: Data Mining Application

18 Sep, 2009 - 05:51 AM
Post #15

New D.I.C Head
*

Joined: 13 Sep, 2008
Posts: 45



QUOTE(Aeternalis @ 16 Sep, 2009 - 12:39 PM) *


Since your work is just for a school project, you wont have to concern yourself with actual implementation scenarios.
Just create your own proprietary formulas that are logical and are based off of the data you are provided



"My own proprietary formulas that are logical " , And how can I know if it's correct formulas or not?

Thanks alot to you.

User is offlineProfile CardPM
+Quote Post

alowais

RE: Data Mining Application

18 Sep, 2009 - 06:00 AM
Post #16

New D.I.C Head
*

Joined: 13 Sep, 2008
Posts: 45

QUOTE(NickDMax @ 16 Sep, 2009 - 01:57 PM) *

....... -- it would be helpful in formulating rules for telling if people who meet certain criteria are a good risk or a bad risk.


Well this is what am asking for, rules that can tell if customer will pay his debts or not?
QUOTE(NickDMax @ 16 Sep, 2009 - 01:57 PM) *

The problem with your idea is getting the data to initially mine.


No the data is not initially mine , but I'm trying to understand how the rules that controls the acceptance or rejection of customers should be?

Thanks
User is offlineProfile CardPM
+Quote Post

alowais

RE: Data Mining Application

18 Sep, 2009 - 06:06 AM
Post #17

New D.I.C Head
*

Joined: 13 Sep, 2008
Posts: 45

QUOTE(Jayman @ 16 Sep, 2009 - 01:07 PM) *

You should be asking these questions from whomever assigned you the project. They are the ones who will define the criteria that will determine the amount of risk.
There is not a specific formula to calculate risk, the formula is specific to the clients needs.


Yes you're right ,but I think there might be a formulas the banks or the lenders use.
User is offlineProfile CardPM
+Quote Post

skyhawk133

RE: Data Mining Application

18 Sep, 2009 - 06:09 AM
Post #18

Head DIC Head
Group Icon

Joined: 17 Mar, 2001
Posts: 16,883



Thanked: 156 times
Dream Kudos: 1650
Expert In: Web Development

My Contributions
Yes, it's called a "credit score": http://en.wikipedia.org/wiki/Credit_score

There are several agencies that provide access to peoples credit scores through an API. You usually have to pay a $$ per query, or block of queries and your application has to meet all their criteria.

What you are looking for though is the Credit Score. The banks then look at the credit score and based on the score, and a few other criteria, they calculate risk. Just google "Credit Score" and you'll come up with lots of info.
User is online!Profile CardPM
+Quote Post

Aeternalis

RE: Data Mining Application

18 Sep, 2009 - 09:29 AM
Post #19

D.I.C Regular
***

Joined: 13 Jul, 2009
Posts: 273



Thanked: 25 times
My Contributions
[[/quote]

"My own proprietary formulas that are logical " , And how can I know if it's correct formulas or not?

Thanks alot to you.
[/quote]

Alowais,
What do you mean by "correct"? There is no Iron Clad way to know if a person is going to pay their bills. The best you can do is estimate the risk of them not paying their bills by comparing their information to a large group of similar information and looking at how many (the percentage) of those people did not pay their bills. Once you have that percentage, (say 10%) you can then say that for 100 people in that situation you are going to have 10 of them not pay the bill.

From their you must figure the cost of them not paying the bill and make sure the interest you earn on the other 90 people is enough to cover the loss, and make the profit you are looking for. If it's not enough.. you will have to raise the interest rate on the 90 people or deny them credit.

The actual formulas used to do this procedure vary. You can do it by designing your own formulas based on what you think is going to produce the results your looking for.

That is what I mean by designing your own proprietary formulas that are logical.

good luck with this, as it can be very complex. Your's doesnt have to be.. you can even make up arbitrary numbers if you don't have a database of information to use to get the percentages from.

Aet
User is offlineProfile CardPM
+Quote Post

alowais

RE: Data Mining Application

19 Sep, 2009 - 08:05 AM
Post #20

New D.I.C Head
*

Joined: 13 Sep, 2008
Posts: 45

QUOTE(Aeternalis @ 18 Sep, 2009 - 09:29 AM) *

The best you can do is estimate the risk of them not paying their bills by comparing their information to a large group of similar information and looking at how many (the percentage) of those people did not pay their bills. Once you have that percentage, (say 10%) you can then say that for 100 people in that situation you are going to have 10 of them not pay the bill.


Great help ,Thanks
User is offlineProfile CardPM
+Quote Post

Fast ReplyReply to this topicStart new topic

Time is now: 11/21/09 02:11PM

Live Help!

Be Social

Dream.In.Code RSS Feed Dream.In.Code LinkedIn Group Follow Us On Twitter Fan Us On Facebook

Tutorials

Programming

Web Development

Reference Sheets

Code Snippets

DIC Chatroom

Bye Bye Ads

Monthly Drawing

Thumb Drive

Top Contributors

Top 10 Kudos This Month