DISEASE CHARACTERISTICS:

• Histologically documented adenocarcinoma of the prostate

• Progressive, systemic (metastatic) disease despite castrate levels of testosterone due to orchiectomy or luteinizing-hormone releasing hormone (LHRH) agonist, meeting 1 of the following criteria:

  • Measurable disease, defined as any lesion that can be accurately measured in at least 1 dimension ≥ 2 cm by conventional techniques or ≥ 1 cm by spiral CT scan or MRI

  • Nonmeasurable disease with PSA ≥ 5 ng/mL

    • Bone lesions
    • Pleural or pericardial effusions, ascites
    • CNS lesions, leptomeningeal disease
    • Irradiated lesions, unless progression documented after radiotherapy
    • No PSA ≥ 5 ng/mL as only evidence of disease
  • PSA evidence for progressive prostate cancer consists of a PSA level ≥ 5 ng/mL that has risen on ≥ 2 successive occasions, ≥ 2 weeks apart

• Castrate levels of testosterone (< 50 ng/dL) must be maintained

  • If no prior orchiectomy, patients must remain on testicular androgen suppression (e.g., with an LHRH analogue)

• Patients receiving an antiandrogen as part of primary androgen ablation must demonstrate disease progression after discontinuation of antiandrogen

  • Disease progression after antiandrogen withdrawal is defined as 2 consecutive rising PSA values, obtained at least 2 weeks apart, or documented osseous or soft tissue progression

    • For patients receiving flutamide or megestrol acetate, at least 1 of the PSA values must be obtained 4 weeks or more after flutamide/megestrol acetate discontinuation
    • For patients receiving bicalutamide or nilutamide, at least 1 of the PSA values must be obtained 6 weeks or more after antiandrogen discontinuation
  • If improvement after antiandrogen withdrawal is noted, disease progression must be established
• No pleural effusion or ascites that causes respiratory compromise ( ≥ grade 2 dyspnea)
• No history of CNS disease, including primary brain tumor, seizures, or carcinomatous meningitis

PATIENT CHARACTERISTICS:

• Fertile patients must use effective barrier contraception during and for 3 months after completion of study treatment
• Karnofsky performance status ≥ 60%
• Life expectancy > 12 weeks
• Granulocyte count > 1,500/mm^3
• Platelet count > 75,000/mm^3
• Hemoglobin > 8.0 g/dL
• Creatinine < 1.5 times upper limit of normal (ULN)
• Bilirubin < 1.5 times ULN
• SGOT/SGPT < 1.5 times ULN
• Urinalysis ≤ 1+ proteinuria based on dipstick reading OR 2+ proteinuria on dipstick reading AND total urinary protein ≤ 3,500 mg on 24 hour urine collection and creatinine clearance ≥ 50 mL/min on a 24-hour urine collection
• No impairment of gastrointestinal (GI) function or GI disease that may affect or alter absorption of vatalanib (i.e., malabsorption syndromes)
• No myocardial infarction or significant change in anginal pattern within the last 6 months, symptomatic congestive heart failure (New York Heart Association class III or IV), or uncontrolled cardiac arrhythmia
• No pre-existing grade 3 or 4 clinical peripheral neuropathy
• No history of severe hypersensitivity reaction to docetaxel or other drugs formulated with polysorbate 80
• No deep vein thrombosis or pulmonary embolus within the past year
• No poorly controlled diabetes (fasting blood glucose > 250) despite optimization of medical therapy
• No labile or poorly controlled hypertension (systolic blood pressure > 160 mm Hg, diastolic blood pressure > 90 mm Hg) despite maximal management with anti-hypertensives

• No serious uncontrolled, concurrent medical illness, including ongoing or active infection

  • Patients on Suppressive antibiotic therapy for chronic urinary tract infection are eligible
• No psychiatric illness or social situation that would limit compliance with treatment

• No "currently active" second malignancy other than nonmelanoma skin cancers

  • Not considered "currently active" if competed therapy and at < 30% risk of relapse
• No interstitial pneumonia or extensive and symptomatic interstitial fibrosis of the lung

PRIOR CONCURRENT THERAPY:

• See Disease Characteristics
• No grapefruit or grapefruit juice during study treatment
• No history of gastrectomy/small bowel resection
• At least 4 weeks since prior hormonal therapy, including ketoconazole, aminoglutethimide, systemic steroids (any dose), and megestrol acetate (any dose)
• At least 4 weeks since prior drug or herbal product known to decrease PSA levels (e.g., finasteride, saw palmetto, or PC-SPES)
• At least 4 weeks since prior major surgery and fully recovered
• At least 4 weeks since prior radiation therapy and fully recovered
• At least 8 weeks since the last dose of prior strontium chloride Sr 89 or samarium Sm 153 lexidronam pentasodium

• Patients receiving bisphosphonate therapy prior to initiating protocol treatment must have received bisphosphonates for at least the past month

  • No bisphosphonate initiation for 1 month prior to and during study treatment
• No prior systemic chemotherapy for prostate cancer
• No prior antiangiogenic agents (thalidomide, bevacizumab)
• No other concurrent chemotherapy, investigational agents, radiotherapy (including palliative), or biologic therapy
• No biologic therapy or immunotherapy ≤ 4 weeks prior to study treatment
• No more than 1 prior therapy with an investigational agent, completed ≥ 4 weeks prior to study treatment
• No concurrent combination antiretroviral therapy for HIV-positive patients

• No concurrent therapeutic warfarin or similar oral anticoagulant that is metabolized by the cytochrome p450 system

  • Heparin is allowed

• No other concurrent hormonal therapy except for the following:

  • Steroids for adrenal failure
  • Hormones for nondisease-related conditions (e.g., insulin for diabetes)
  • Intermittent dexamethasone